2024-05-08T22:14:03.033 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T22:14:03.033 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/direct_io}} --verbose --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109 -- /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109/orig.config.yaml 2024-05-08T22:14:03.088 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T22:14:03.183 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109 branch: wip-pdonnell-testing-20240508.150423-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/direct_io}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698109' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240508.150423-reef ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.150423-reef sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2738 sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 sleep_before_teardown: 0 subset: 17/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.150423-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDfkUiyItDwxZ6VNw2sSrcLaaL0b721fNvh+/CBn1ns4cFiSjSXTQ0sC0AaoKL/YJNyW94n29ATRx9syHVUaYk0= smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxBdlTbVMsxxBuQe4xUOnXgCDLKA8SaHvZ64uVyLk81zCdh5jHbrZYPcsZeaZ47WMGfechHM/e3bq85KrS9y+M= smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG32x3mUp2tjxN6mAX54sDcvMPvA51cyVHu+flBjQVcKmPxe/4VFzeyH9rLJZuZIGYHNOyyzPvG47rnJXE1wfQM= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_19:22:21 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T22:14:03.183 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa; will attempt to use it 2024-05-08T22:14:03.184 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks 2024-05-08T22:14:03.184 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T22:14:03.185 INFO:teuthology.task.internal:Checking packages... 2024-05-08T22:14:03.208 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'd21e4f961318b56f0d915caa2490c3711bf13c34' 2024-05-08T22:14:03.208 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T22:14:03.208 INFO:teuthology.packaging:ref: None 2024-05-08T22:14:03.208 INFO:teuthology.packaging:tag: None 2024-05-08T22:14:03.208 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.150423-reef 2024-05-08T22:14:03.208 INFO:teuthology.packaging:sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:14:03.208 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240508.150423-reef 2024-05-08T22:14:03.353 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1322.gd21e4f96 2024-05-08T22:14:03.355 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T22:14:03.362 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T22:14:03.362 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T22:14:03.406 INFO:teuthology.task.internal:Saving configuration 2024-05-08T22:14:03.426 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T22:14:03.433 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T22:14:03.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:07:56.692084', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDfkUiyItDwxZ6VNw2sSrcLaaL0b721fNvh+/CBn1ns4cFiSjSXTQ0sC0AaoKL/YJNyW94n29ATRx9syHVUaYk0='} 2024-05-08T22:14:03.480 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:07:56.694981', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxBdlTbVMsxxBuQe4xUOnXgCDLKA8SaHvZ64uVyLk81zCdh5jHbrZYPcsZeaZ47WMGfechHM/e3bq85KrS9y+M='} 2024-05-08T22:14:03.502 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:07:56.693774', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG32x3mUp2tjxN6mAX54sDcvMPvA51cyVHu+flBjQVcKmPxe/4VFzeyH9rLJZuZIGYHNOyyzPvG47rnJXE1wfQM='} 2024-05-08T22:14:03.502 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T22:14:03.511 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T22:14:03.511 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T22:14:03.511 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T22:14:03.511 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T22:14:03.641 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f397dd67160>, signals=[15]) 2024-05-08T22:14:03.642 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T22:14:03.648 INFO:teuthology.task.internal:Opening connections... 2024-05-08T22:14:03.648 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-05-08T22:14:03.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:14:03.733 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-05-08T22:14:03.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:14:03.816 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2024-05-08T22:14:03.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:14:03.896 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T22:14:03.903 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-05-08T22:14:03.925 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-05-08T22:14:03.926 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi062.stdout:ID="rhel" 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="fedora" 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8.6" 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T22:14:03.986 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T22:14:03.987 INFO:teuthology.orchestra.run.smithi062.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T22:14:03.987 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T22:14:03.987 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:14:03.987 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T22:14:03.987 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T22:14:03.987 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T22:14:03.987 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T22:14:03.988 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-05-08T22:14:04.009 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-05-08T22:14:04.031 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-05-08T22:14:04.031 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-05-08T22:14:04.092 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T22:14:04.092 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T22:14:04.092 INFO:teuthology.orchestra.run.smithi066.stdout:ID="rhel" 2024-05-08T22:14:04.092 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="fedora" 2024-05-08T22:14:04.092 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="8.6" 2024-05-08T22:14:04.092 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el8" 2024-05-08T22:14:04.092 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T22:14:04.092 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2024-05-08T22:14:04.092 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T22:14:04.093 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T22:14:04.093 INFO:teuthology.orchestra.run.smithi066.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T22:14:04.093 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T22:14:04.093 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:14:04.093 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T22:14:04.093 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T22:14:04.093 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T22:14:04.093 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T22:14:04.094 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-05-08T22:14:04.117 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-05-08T22:14:04.138 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-05-08T22:14:04.138 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-05-08T22:14:04.197 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout:ID="rhel" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="fedora" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="8.6" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el8" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T22:14:04.198 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:14:04.199 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T22:14:04.199 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T22:14:04.199 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T22:14:04.199 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T22:14:04.199 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-05-08T22:14:04.216 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T22:14:04.228 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T22:14:04.233 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T22:14:04.234 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-05-08T22:14:04.236 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-05-08T22:14:04.238 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2024-05-08T22:14:04.256 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T22:14:04.263 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T22:14:04.263 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-05-08T22:14:04.296 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-05-08T22:14:04.299 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2024-05-08T22:14:04.324 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T22:14:04.405 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T22:14:04.420 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-08T22:14:04.421 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-08T22:14:04.421 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-08T22:14:04.421 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T22:14:04.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:14:04.441 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-05-08T22:14:04.503 INFO:teuthology.orchestra.run.smithi062.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:14:04.504 INFO:teuthology.task.kernel:Running kernel on smithi062: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:14:04.504 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum install -y kernel 2024-05-08T22:14:06.065 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2024-05-08T22:14:06.065 INFO:teuthology.orchestra.run.smithi062.stdout:Unable to read consumer identity 2024-05-08T22:14:06.320 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:14:06.320 INFO:teuthology.orchestra.run.smithi062.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:14:06.320 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:14:07.995 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2024-05-08T22:14:08.838 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2024-05-08T22:14:12.663 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 410 kB/s | 24 kB 00:00 2024-05-08T22:14:13.441 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T22:14:13.441 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T22:14:13.544 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-05-08T22:14:13.545 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-05-08T22:14:13.545 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-05-08T22:14:13.681 DEBUG:teuthology.orchestra.run.smithi062:> echo no | sudo yum reinstall kernel || true 2024-05-08T22:14:14.037 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2024-05-08T22:14:14.037 INFO:teuthology.orchestra.run.smithi062.stdout:Unable to read consumer identity 2024-05-08T22:14:14.192 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:14:14.192 INFO:teuthology.orchestra.run.smithi062.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:14:14.192 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:14:14.518 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 May 2024 10:14:12 PM UTC. 2024-05-08T22:14:14.605 INFO:teuthology.orchestra.run.smithi062.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T22:14:14.612 INFO:teuthology.orchestra.run.smithi062.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T22:14:14.650 INFO:teuthology.orchestra.run.smithi062.stderr:Error: No packages marked for reinstall. 2024-05-08T22:14:14.698 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum reinstall -y kernel || true 2024-05-08T22:14:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2024-05-08T22:14:15.049 INFO:teuthology.orchestra.run.smithi062.stdout:Unable to read consumer identity 2024-05-08T22:14:15.237 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:14:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:14:15.238 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:14:15.554 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:00:03 ago on Wed 08 May 2024 10:14:12 PM UTC. 2024-05-08T22:14:15.626 INFO:teuthology.orchestra.run.smithi062.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T22:14:15.626 INFO:teuthology.orchestra.run.smithi062.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T22:14:15.655 INFO:teuthology.orchestra.run.smithi062.stderr:Error: No packages marked for reinstall. 2024-05-08T22:14:15.703 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T22:14:15.861 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:14:15.861 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:14:15.861 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T22:14:15.861 DEBUG:teuthology.parallel:result is None 2024-05-08T22:14:15.861 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T22:14:15.868 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T22:14:15.869 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T22:14:15.871 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T22:14:15.874 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T22:14:15.894 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T22:14:15.900 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T22:14:15.908 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T22:14:15.908 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T22:14:15.928 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T22:14:15.935 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T22:14:15.971 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T22:14:15.978 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T22:14:15.978 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T22:14:16.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:14:16.017 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T22:14:16.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:14:16.035 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T22:14:16.037 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T22:14:16.060 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T22:14:16.079 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:14:16.094 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:14:16.105 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:14:16.116 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:14:16.122 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:14:16.155 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:14:16.157 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T22:14:16.164 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T22:14:16.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T22:14:16.167 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T22:14:16.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T22:14:16.227 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T22:14:16.236 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T22:14:16.236 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T22:14:16.238 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T22:14:16.241 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T22:14:16.286 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:14:16.336 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:14:16.393 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:14:16.479 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:14:16.520 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:14:16.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T22:14:16.586 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:14:16.635 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:14:16.691 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:14:16.776 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:14:16.824 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-08T22:14:16.824 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T22:14:16.893 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:14:16.940 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:14:16.998 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:14:17.084 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:14:17.127 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-08T22:14:17.127 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T22:14:17.195 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-05-08T22:14:17.198 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-05-08T22:14:17.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2024-05-08T22:14:17.260 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:14:17.270 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:14:17.298 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:14:17.687 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T22:14:17.695 INFO:teuthology.task.internal:Starting timer... 2024-05-08T22:14:17.695 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T22:14:17.705 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T22:14:17.713 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2024-05-08T22:14:17.714 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-05-08T22:14:17.774 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-05-08T22:14:17.776 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2024-05-08T22:14:17.834 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2024-05-08T22:14:17.837 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2024-05-08T22:14:17.902 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2024-05-08T22:14:17.904 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T22:14:17.904 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-05-08T22:14:17.932 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-05-08T22:14:17.932 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2024-05-08T22:14:17.953 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2024-05-08T22:14:17.953 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2024-05-08T22:14:17.973 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2024-05-08T22:14:17.973 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi079.front.sepia.ceph.com': 'permissive'} 2024-05-08T22:14:17.973 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:14:18.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:14:18.001 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:14:18.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:14:18.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:14:18.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:14:18.056 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T22:14:18.056 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-05-08T22:14:18.083 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2024-05-08T22:14:18.114 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2024-05-08T22:14:18.149 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T22:14:18.159 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T22:14:18.175 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T22:14:18.177 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi066.front.sepia.ceph.com,smithi079.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T22:24:42.732 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi079.front.sepia.ceph.com')] 2024-05-08T22:24:42.733 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-05-08T22:24:42.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:24:42.813 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-05-08T22:24:42.881 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-05-08T22:24:42.881 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-05-08T22:24:42.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:24:42.960 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-05-08T22:24:43.030 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-05-08T22:24:43.031 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-05-08T22:24:43.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:24:43.106 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-05-08T22:24:43.176 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-05-08T22:24:43.177 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T22:24:43.188 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T22:24:43.188 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T22:24:43.188 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:24:43.191 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T22:24:43.191 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:24:43.193 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T22:24:43.194 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:24:43.227 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T22:24:43.229 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T22:24:43.245 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T22:24:43.247 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T22:24:43.251 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T22:24:43.266 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T22:24:43.321 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2024-05-08T22:24:43.328 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-05-08T22:24:43.334 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2024-05-08T22:24:43.341 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-05-08T22:24:43.343 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2024-05-08T22:24:43.350 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T22:24:43.354 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2024-05-08T22:24:43.357 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T22:24:43.366 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T22:24:43.369 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T22:24:43.372 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T22:24:43.383 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T22:24:43.439 INFO:teuthology.orchestra.run.smithi066.stderr:bash: ntpq: command not found 2024-05-08T22:24:43.444 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:24:43.444 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-05-08T22:24:43.444 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.444 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.444 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.444 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.448 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2024-05-08T22:24:43.449 INFO:teuthology.orchestra.run.smithi079.stderr:bash: ntpq: command not found 2024-05-08T22:24:43.453 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:24:43.453 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-05-08T22:24:43.453 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.453 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.453 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.453 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.455 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:24:43.455 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-05-08T22:24:43.455 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.455 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.455 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.455 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:24:43.455 INFO:teuthology.run_tasks:Running task pexec... 2024-05-08T22:24:43.465 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-08T22:24:43.465 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi062.front.sepia.ceph.com 2024-05-08T22:24:43.465 DEBUG:teuthology.orchestra.run.smithi062:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T22:24:43.466 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi066.front.sepia.ceph.com 2024-05-08T22:24:43.466 DEBUG:teuthology.orchestra.run.smithi066:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T22:24:43.466 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi079.front.sepia.ceph.com 2024-05-08T22:24:43.466 DEBUG:teuthology.orchestra.run.smithi079:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T22:24:43.870 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2024-05-08T22:24:43.883 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2024-05-08T22:24:43.889 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2024-05-08T22:24:45.669 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 65 kB/s | 4.5 kB 00:00 2024-05-08T22:24:45.909 INFO:teuthology.orchestra.run.smithi079.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-05-08T22:24:45.978 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-05-08T22:24:46.224 INFO:teuthology.orchestra.run.smithi079.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 kB/s | 4.5 kB 00:00 2024-05-08T22:24:46.275 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-05-08T22:24:46.548 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 53 kB/s | 4.1 kB 00:00 2024-05-08T22:24:49.360 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-05-08T22:24:49.361 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-05-08T22:24:49.361 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-05-08T22:24:49.361 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-05-08T22:24:49.361 INFO:teuthology.orchestra.run.smithi079.stdout:Resetting modules: 2024-05-08T22:24:49.361 INFO:teuthology.orchestra.run.smithi079.stdout: container-tools 2024-05-08T22:24:49.361 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:24:49.362 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-05-08T22:24:49.362 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-05-08T22:24:49.362 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:24:49.547 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-05-08T22:24:49.548 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-05-08T22:24:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-05-08T22:24:49.548 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-05-08T22:24:49.549 INFO:teuthology.orchestra.run.smithi066.stdout:Resetting modules: 2024-05-08T22:24:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: container-tools 2024-05-08T22:24:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:24:49.549 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-05-08T22:24:49.549 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-05-08T22:24:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:24:49.827 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-05-08T22:24:49.828 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-05-08T22:24:49.828 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-05-08T22:24:49.828 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-05-08T22:24:49.828 INFO:teuthology.orchestra.run.smithi062.stdout:Resetting modules: 2024-05-08T22:24:49.828 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools 2024-05-08T22:24:49.829 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:24:49.829 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-05-08T22:24:49.829 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-05-08T22:24:49.829 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:25:04.565 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-05-08T22:25:05.036 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2024-05-08T22:25:05.200 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-05-08T22:25:05.202 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-05-08T22:25:05.673 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2024-05-08T22:25:05.710 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2024-05-08T22:25:06.976 INFO:teuthology.orchestra.run.smithi079.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 kB/s | 4.1 kB 00:00 2024-05-08T22:25:07.290 INFO:teuthology.orchestra.run.smithi079.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 kB/s | 4.5 kB 00:00 2024-05-08T22:25:07.505 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-05-08T22:25:07.636 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-05-08T22:25:08.106 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-05-08T22:25:08.244 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-05-08T22:25:10.190 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-05-08T22:25:10.352 INFO:teuthology.orchestra.run.smithi079.stdout:========================================================================================================== 2024-05-08T22:25:10.352 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-05-08T22:25:10.352 INFO:teuthology.orchestra.run.smithi079.stdout:========================================================================================================== 2024-05-08T22:25:10.352 INFO:teuthology.orchestra.run.smithi079.stdout:Installing group/module packages: 2024-05-08T22:25:10.352 INFO:teuthology.orchestra.run.smithi079.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T22:25:10.352 INFO:teuthology.orchestra.run.smithi079.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T22:25:10.352 INFO:teuthology.orchestra.run.smithi079.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T22:25:10.352 INFO:teuthology.orchestra.run.smithi079.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T22:25:10.352 INFO:teuthology.orchestra.run.smithi079.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:25:10.352 INFO:teuthology.orchestra.run.smithi079.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout:Installing weak dependencies: 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout:Installing module profiles: 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: container-tools/common 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout:Enabling module streams: 2024-05-08T22:25:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: container-tools rhel8 2024-05-08T22:25:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:25:10.354 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-05-08T22:25:10.354 INFO:teuthology.orchestra.run.smithi079.stdout:========================================================================================================== 2024-05-08T22:25:10.354 INFO:teuthology.orchestra.run.smithi079.stdout:Install 13 Packages 2024-05-08T22:25:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:25:10.354 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 26 M 2024-05-08T22:25:10.354 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 87 M 2024-05-08T22:25:10.355 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-05-08T22:25:10.474 INFO:teuthology.orchestra.run.smithi079.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 208 kB/s | 25 kB 00:00 2024-05-08T22:25:10.499 INFO:teuthology.orchestra.run.smithi079.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 652 kB/s | 94 kB 00:00 2024-05-08T22:25:10.541 INFO:teuthology.orchestra.run.smithi079.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 211 kB/s | 39 kB 00:00 2024-05-08T22:25:10.616 INFO:teuthology.orchestra.run.smithi079.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 288 kB/s | 41 kB 00:00 2024-05-08T22:25:10.658 INFO:teuthology.orchestra.run.smithi079.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 6.4 MB/s | 1.0 MB 00:00 2024-05-08T22:25:10.700 INFO:teuthology.orchestra.run.smithi079.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 8.6 MB/s | 739 kB 00:00 2024-05-08T22:25:10.758 INFO:teuthology.orchestra.run.smithi079.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 2.3 MB/s | 239 kB 00:00 2024-05-08T22:25:10.859 INFO:teuthology.orchestra.run.smithi079.stdout:(8/13): netavark-1.7.0-2.module+el8.9.0+21697+6 23 MB/s | 3.7 MB 00:00 2024-05-08T22:25:10.884 INFO:teuthology.orchestra.run.smithi079.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 1.2 MB/s | 154 kB 00:00 2024-05-08T22:25:11.010 INFO:teuthology.orchestra.run.smithi079.stdout:(10/13): buildah-1.31.5-1.module+el8.9.0+21697+ 19 MB/s | 8.8 MB 00:00 2024-05-08T22:25:11.093 INFO:teuthology.orchestra.run.smithi079.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 12 MB/s | 2.5 MB 00:00 2024-05-08T22:25:11.127 INFO:teuthology.orchestra.run.smithi079.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 423 kB/s | 49 kB 00:00 2024-05-08T22:25:11.252 INFO:teuthology.orchestra.run.smithi079.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 21 MB/s | 8.1 MB 00:00 2024-05-08T22:25:11.253 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:25:11.253 INFO:teuthology.orchestra.run.smithi079.stdout:Total 28 MB/s | 26 MB 00:00 2024-05-08T22:25:11.327 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-05-08T22:25:11.408 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-05-08T22:25:11.502 INFO:teuthology.orchestra.run.smithi062.stdout:========================================================================================================== 2024-05-08T22:25:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-05-08T22:25:11.502 INFO:teuthology.orchestra.run.smithi062.stdout:========================================================================================================== 2024-05-08T22:25:11.502 INFO:teuthology.orchestra.run.smithi062.stdout:Installing group/module packages: 2024-05-08T22:25:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T22:25:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T22:25:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T22:25:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T22:25:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout:Installing module profiles: 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools/common 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:25:11.503 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling module streams: 2024-05-08T22:25:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools rhel8 2024-05-08T22:25:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:25:11.504 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-05-08T22:25:11.504 INFO:teuthology.orchestra.run.smithi062.stdout:========================================================================================================== 2024-05-08T22:25:11.504 INFO:teuthology.orchestra.run.smithi062.stdout:Install 13 Packages 2024-05-08T22:25:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:25:11.504 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 26 M 2024-05-08T22:25:11.505 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 87 M 2024-05-08T22:25:11.505 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-05-08T22:25:11.528 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-05-08T22:25:11.550 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-05-08T22:25:11.550 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-05-08T22:25:11.583 INFO:teuthology.orchestra.run.smithi066.stdout:========================================================================================================== 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout:========================================================================================================== 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout:Installing group/module packages: 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-05-08T22:25:11.584 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout:Installing weak dependencies: 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout:Installing module profiles: 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout: container-tools/common 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling module streams: 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout: container-tools rhel8 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout:========================================================================================================== 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout:Install 13 Packages 2024-05-08T22:25:11.585 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:25:11.586 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 26 M 2024-05-08T22:25:11.586 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 87 M 2024-05-08T22:25:11.586 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-05-08T22:25:11.680 INFO:teuthology.orchestra.run.smithi062.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 142 kB/s | 25 kB 00:00 2024-05-08T22:25:11.705 INFO:teuthology.orchestra.run.smithi062.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 197 kB/s | 39 kB 00:00 2024-05-08T22:25:11.730 INFO:teuthology.orchestra.run.smithi062.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 417 kB/s | 94 kB 00:00 2024-05-08T22:25:11.759 INFO:teuthology.orchestra.run.smithi066.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 144 kB/s | 25 kB 00:00 2024-05-08T22:25:11.778 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-05-08T22:25:11.787 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-05-08T22:25:11.792 INFO:teuthology.orchestra.run.smithi066.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 191 kB/s | 39 kB 00:00 2024-05-08T22:25:11.851 INFO:teuthology.orchestra.run.smithi066.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 355 kB/s | 94 kB 00:00 2024-05-08T22:25:11.880 INFO:teuthology.orchestra.run.smithi062.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 204 kB/s | 41 kB 00:00 2024-05-08T22:25:11.972 INFO:teuthology.orchestra.run.smithi062.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 3.8 MB/s | 1.0 MB 00:00 2024-05-08T22:25:11.976 INFO:teuthology.orchestra.run.smithi066.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 5.5 MB/s | 1.0 MB 00:00 2024-05-08T22:25:12.002 INFO:teuthology.orchestra.run.smithi066.stdout:(5/13): yajl-2.1.0-12.el8.x86_64.rpm 169 kB/s | 41 kB 00:00 2024-05-08T22:25:12.014 INFO:teuthology.orchestra.run.smithi062.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 5.4 MB/s | 739 kB 00:00 2024-05-08T22:25:12.085 INFO:teuthology.orchestra.run.smithi066.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 6.6 MB/s | 739 kB 00:00 2024-05-08T22:25:12.131 INFO:teuthology.orchestra.run.smithi062.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 22 MB/s | 8.8 MB 00:00 2024-05-08T22:25:12.156 INFO:teuthology.orchestra.run.smithi062.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.3 MB/s | 239 kB 00:00 2024-05-08T22:25:12.219 INFO:teuthology.orchestra.run.smithi066.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 24 MB/s | 8.8 MB 00:00 2024-05-08T22:25:12.244 INFO:teuthology.orchestra.run.smithi066.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 988 kB/s | 239 kB 00:00 2024-05-08T22:25:12.257 INFO:teuthology.orchestra.run.smithi062.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 15 MB/s | 3.7 MB 00:00 2024-05-08T22:25:12.291 INFO:teuthology.orchestra.run.smithi062.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 967 kB/s | 154 kB 00:00 2024-05-08T22:25:12.336 INFO:teuthology.orchestra.run.smithi066.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 15 MB/s | 3.7 MB 00:00 2024-05-08T22:25:12.361 INFO:teuthology.orchestra.run.smithi066.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 1.1 MB/s | 154 kB 00:00 2024-05-08T22:25:12.474 INFO:teuthology.orchestra.run.smithi062.stdout:(11/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 26 MB/s | 8.1 MB 00:00 2024-05-08T22:25:12.499 INFO:teuthology.orchestra.run.smithi062.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 237 kB/s | 49 kB 00:00 2024-05-08T22:25:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-05-08T22:25:12.550 INFO:teuthology.orchestra.run.smithi062.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 8.7 MB/s | 2.5 MB 00:00 2024-05-08T22:25:12.550 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:25:12.550 INFO:teuthology.orchestra.run.smithi062.stdout:Total 24 MB/s | 26 MB 00:01 2024-05-08T22:25:12.553 INFO:teuthology.orchestra.run.smithi066.stdout:(11/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 26 MB/s | 8.1 MB 00:00 2024-05-08T22:25:12.595 INFO:teuthology.orchestra.run.smithi066.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 212 kB/s | 49 kB 00:00 2024-05-08T22:25:12.670 INFO:teuthology.orchestra.run.smithi066.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 7.6 MB/s | 2.5 MB 00:00 2024-05-08T22:25:12.671 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:25:12.671 INFO:teuthology.orchestra.run.smithi066.stdout:Total 24 MB/s | 26 MB 00:01 2024-05-08T22:25:12.838 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-05-08T22:25:12.861 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-05-08T22:25:12.861 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-05-08T22:25:12.962 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-05-08T22:25:12.985 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-05-08T22:25:12.985 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-05-08T22:25:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T22:25:13.105 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-05-08T22:25:13.115 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-05-08T22:25:13.230 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-05-08T22:25:13.239 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-05-08T22:25:13.242 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T22:25:13.396 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T22:25:13.555 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T22:25:13.735 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T22:25:13.857 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-05-08T22:25:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T22:25:13.989 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-05-08T22:25:14.179 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T22:25:14.355 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T22:25:14.508 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T22:25:14.523 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T22:25:14.694 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T22:25:14.699 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T22:25:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T22:25:14.852 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T22:25:15.006 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T22:25:15.021 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T22:25:15.076 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:25:15.183 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T22:25:15.237 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T22:25:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:25:15.407 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T22:25:15.637 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T22:25:15.642 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T22:25:15.688 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T22:25:16.558 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:25:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:25:16.580 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T22:25:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:25:16.791 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T22:25:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:25:16.974 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:25:17.174 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T22:25:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T22:25:17.379 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:25:17.379 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T22:25:17.379 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T22:25:17.379 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T22:25:17.379 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T22:25:17.379 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T22:25:17.379 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T22:25:17.380 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T22:25:17.380 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:25:17.380 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:25:17.380 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T22:25:17.380 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T22:25:17.380 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T22:25:18.107 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T22:25:18.153 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T22:25:18.316 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T22:25:18.346 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T22:25:18.502 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:25:18.505 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:25:18.926 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:25:18.926 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T22:25:18.926 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T22:25:18.927 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T22:25:18.927 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T22:25:18.927 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T22:25:18.927 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T22:25:18.927 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T22:25:18.927 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:25:18.927 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:25:18.927 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T22:25:18.927 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T22:25:18.927 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T22:25:18.934 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T22:25:18.934 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T22:25:18.934 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T22:25:18.934 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T22:25:18.934 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T22:25:18.934 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T22:25:18.934 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T22:25:18.934 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T22:25:18.934 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T22:25:18.934 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T22:25:18.934 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T22:25:18.935 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T22:25:18.935 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T22:25:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T22:25:27.657 INFO:teuthology.orchestra.run.smithi079.stdout:Installed products updated. 2024-05-08T22:25:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:25:28.231 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-05-08T22:25:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:25:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T22:25:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:25:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T22:25:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T22:25:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:25:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T22:25:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:25:28.232 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-05-08T22:25:28.388 DEBUG:teuthology.parallel:result is None 2024-05-08T22:25:30.072 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T22:25:30.072 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2024-05-08T22:25:30.684 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:25:30.684 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-05-08T22:25:30.684 INFO:teuthology.orchestra.run.smithi062.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:30.684 INFO:teuthology.orchestra.run.smithi062.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:25:30.685 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-05-08T22:25:30.855 DEBUG:teuthology.parallel:result is None 2024-05-08T22:25:31.422 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T22:25:31.422 INFO:teuthology.orchestra.run.smithi066.stdout:Installed products updated. 2024-05-08T22:25:32.040 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T22:25:32.041 INFO:teuthology.orchestra.run.smithi066.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T22:25:32.042 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:25:32.042 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-05-08T22:25:32.212 DEBUG:teuthology.parallel:result is None 2024-05-08T22:25:32.213 INFO:teuthology.run_tasks:Running task install... 2024-05-08T22:25:32.224 DEBUG:teuthology.task.install:project ceph 2024-05-08T22:25:32.225 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}} 2024-05-08T22:25:32.225 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'} 2024-05-08T22:25:32.225 INFO:teuthology.task.install:Using flavor: default 2024-05-08T22:25:32.235 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T22:25:32.235 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T22:25:32.235 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T22:25:32.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:25:32.239 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T22:25:32.239 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:25:32.240 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T22:25:32.240 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:25:32.395 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T22:25:32.395 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T22:25:32.397 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T22:25:32.397 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T22:25:32.402 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T22:25:32.402 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T22:25:32.566 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T22:25:32.566 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-08T22:25:32.566 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T22:25:32.574 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T22:25:32.574 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-08T22:25:32.574 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T22:25:32.577 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T22:25:32.577 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:25:32.577 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T22:25:32.596 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T22:25:32.596 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T22:25:32.607 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T22:25:32.607 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T22:25:32.612 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T22:25:32.612 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T22:25:32.665 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T22:25:32.679 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T22:25:32.685 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T22:25:32.744 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T22:25:32.759 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T22:25:32.767 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T22:25:32.811 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2024-05-08T22:25:32.813 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2024-05-08T22:25:32.826 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2024-05-08T22:25:32.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2024-05-08T22:25:32.837 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-05-08T22:25:32.839 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-05-08T22:25:33.206 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2024-05-08T22:25:33.209 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2024-05-08T22:25:33.236 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2024-05-08T22:25:34.695 INFO:teuthology.orchestra.run.smithi079.stdout:63 files removed 2024-05-08T22:25:34.707 INFO:teuthology.orchestra.run.smithi066.stdout:63 files removed 2024-05-08T22:25:34.709 INFO:teuthology.orchestra.run.smithi062.stdout:63 files removed 2024-05-08T22:25:34.747 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T22:25:34.760 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T22:25:34.767 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T22:25:35.085 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2024-05-08T22:25:35.118 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2024-05-08T22:25:35.124 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2024-05-08T22:25:36.973 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 223 kB/s | 80 kB 00:00 2024-05-08T22:25:36.997 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 310 kB/s | 80 kB 00:00 2024-05-08T22:25:37.050 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 227 kB/s | 80 kB 00:00 2024-05-08T22:25:37.331 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 39 kB/s | 11 kB 00:00 2024-05-08T22:25:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-05-08T22:25:37.426 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 34 kB/s | 11 kB 00:00 2024-05-08T22:25:37.649 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 5.6 kB/s | 1.5 kB 00:00 2024-05-08T22:25:37.657 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 5.0 kB/s | 1.5 kB 00:00 2024-05-08T22:25:37.817 INFO:teuthology.orchestra.run.smithi079.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-05-08T22:25:37.818 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2024-05-08T22:25:37.824 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-05-08T22:25:37.969 INFO:teuthology.orchestra.run.smithi066.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2024-05-08T22:25:38.748 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2024-05-08T22:25:38.767 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 18 MB/s | 16 MB 00:00 2024-05-08T22:25:38.960 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2024-05-08T22:25:42.401 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 432 kB/s | 24 kB 00:00 2024-05-08T22:25:42.637 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 453 kB/s | 24 kB 00:00 2024-05-08T22:25:42.842 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 617 kB/s | 24 kB 00:00 2024-05-08T22:25:43.144 INFO:teuthology.orchestra.run.smithi079.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-05-08T22:25:43.480 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 12 MB/s | 8.8 MB 00:00 2024-05-08T22:25:43.577 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-05-08T22:25:45.879 INFO:teuthology.orchestra.run.smithi079.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 55 MB/s | 67 MB 00:01 2024-05-08T22:25:46.615 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 61 MB 00:01 2024-05-08T22:25:47.012 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 MB/s | 61 MB 00:01 2024-05-08T22:25:56.697 INFO:teuthology.orchestra.run.smithi079.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 MB/s | 61 MB 00:01 2024-05-08T22:25:59.266 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 67 MB 00:01 2024-05-08T22:25:59.647 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 67 MB 00:01 2024-05-08T22:26:12.434 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:26:12.435 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:26:12.454 INFO:teuthology.orchestra.run.smithi079.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T22:26:12.455 INFO:teuthology.orchestra.run.smithi079.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T22:26:12.456 INFO:teuthology.orchestra.run.smithi079.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T22:26:12.458 INFO:teuthology.orchestra.run.smithi079.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T22:26:12.460 INFO:teuthology.orchestra.run.smithi079.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T22:26:12.464 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T22:26:12.465 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T22:26:12.680 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-05-08T22:26:12.695 INFO:teuthology.orchestra.run.smithi079.stdout:=================================================================================================================================== 2024-05-08T22:26:12.695 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-05-08T22:26:12.695 INFO:teuthology.orchestra.run.smithi079.stdout:=================================================================================================================================== 2024-05-08T22:26:12.695 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-05-08T22:26:12.695 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T22:26:12.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T22:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T22:26:12.698 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T22:26:12.698 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T22:26:12.698 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T22:26:12.698 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T22:26:12.698 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T22:26:12.698 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T22:26:12.698 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T22:26:12.698 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T22:26:12.698 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T22:26:12.699 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T22:26:12.700 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T22:26:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T22:26:12.702 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T22:26:12.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:26:12.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T22:26:12.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T22:26:12.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T22:26:12.707 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T22:26:12.707 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T22:26:12.707 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T22:26:12.707 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T22:26:12.707 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T22:26:12.707 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T22:26:12.707 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T22:26:12.707 INFO:teuthology.orchestra.run.smithi079.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T22:26:12.707 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T22:26:12.707 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T22:26:12.708 INFO:teuthology.orchestra.run.smithi079.stdout:Enabling module streams: 2024-05-08T22:26:12.708 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql 10 2024-05-08T22:26:12.708 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:26:12.708 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-05-08T22:26:12.708 INFO:teuthology.orchestra.run.smithi079.stdout:=================================================================================================================================== 2024-05-08T22:26:12.708 INFO:teuthology.orchestra.run.smithi079.stdout:Install 130 Packages 2024-05-08T22:26:12.708 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 16 Packages 2024-05-08T22:26:12.708 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:26:12.709 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 215 M 2024-05-08T22:26:12.709 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-05-08T22:26:12.896 INFO:teuthology.orchestra.run.smithi079.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 50 kB/s | 6.4 kB 00:00 2024-05-08T22:26:13.129 INFO:teuthology.orchestra.run.smithi079.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 3.6 MB/s | 860 kB 00:00 2024-05-08T22:26:13.196 INFO:teuthology.orchestra.run.smithi079.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 2.4 MB/s | 166 kB 00:00 2024-05-08T22:26:13.280 INFO:teuthology.orchestra.run.smithi079.stdout:(4/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 11 MB/s | 5.5 MB 00:00 2024-05-08T22:26:13.422 INFO:teuthology.orchestra.run.smithi079.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 11 MB/s | 1.6 MB 00:00 2024-05-08T22:26:13.522 INFO:teuthology.orchestra.run.smithi079.stdout:(6/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 7.0 MB/s | 2.3 MB 00:00 2024-05-08T22:26:13.872 INFO:teuthology.orchestra.run.smithi079.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 11 MB/s | 4.8 MB 00:00 2024-05-08T22:26:15.439 INFO:teuthology.orchestra.run.smithi079.stdout:(8/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x86 9.6 MB/s | 18 MB 00:01 2024-05-08T22:26:15.598 INFO:teuthology.orchestra.run.smithi079.stdout:(9/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el8 7.6 MB/s | 13 MB 00:01 2024-05-08T22:26:15.615 INFO:teuthology.orchestra.run.smithi079.stdout:(10/146): ceph-selinux-18.2.2-1322.gd21e4f96.el 139 kB/s | 24 kB 00:00 2024-05-08T22:26:15.665 INFO:teuthology.orchestra.run.smithi079.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 5.1 MB/s | 261 kB 00:00 2024-05-08T22:26:15.715 INFO:teuthology.orchestra.run.smithi079.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 621 kB/s | 31 kB 00:00 2024-05-08T22:26:15.824 INFO:teuthology.orchestra.run.smithi079.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 7.0 MB/s | 782 kB 00:00 2024-05-08T22:26:15.826 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:26:15.827 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:26:15.846 INFO:teuthology.orchestra.run.smithi066.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T22:26:15.848 INFO:teuthology.orchestra.run.smithi066.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T22:26:15.849 INFO:teuthology.orchestra.run.smithi066.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T22:26:15.851 INFO:teuthology.orchestra.run.smithi066.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T22:26:15.854 INFO:teuthology.orchestra.run.smithi066.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T22:26:15.858 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T22:26:15.860 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T22:26:15.891 INFO:teuthology.orchestra.run.smithi079.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 2.1 MB/s | 142 kB 00:00 2024-05-08T22:26:15.941 INFO:teuthology.orchestra.run.smithi079.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 2.7 MB/s | 139 kB 00:00 2024-05-08T22:26:16.008 INFO:teuthology.orchestra.run.smithi079.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 7.0 MB/s | 474 kB 00:00 2024-05-08T22:26:16.101 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-05-08T22:26:16.103 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:26:16.105 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T22:26:16.116 INFO:teuthology.orchestra.run.smithi066.stdout:=================================================================================================================================== 2024-05-08T22:26:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-05-08T22:26:16.116 INFO:teuthology.orchestra.run.smithi066.stdout:=================================================================================================================================== 2024-05-08T22:26:16.116 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T22:26:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T22:26:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T22:26:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T22:26:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T22:26:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T22:26:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T22:26:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T22:26:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T22:26:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T22:26:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T22:26:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T22:26:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T22:26:16.119 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T22:26:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T22:26:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T22:26:16.121 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-05-08T22:26:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T22:26:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T22:26:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T22:26:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T22:26:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T22:26:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T22:26:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T22:26:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T22:26:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T22:26:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T22:26:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T22:26:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T22:26:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T22:26:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T22:26:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T22:26:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T22:26:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T22:26:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T22:26:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T22:26:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T22:26:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T22:26:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T22:26:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling module streams: 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql 10 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout:=================================================================================================================================== 2024-05-08T22:26:16.128 INFO:teuthology.orchestra.run.smithi066.stdout:Install 130 Packages 2024-05-08T22:26:16.129 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 16 Packages 2024-05-08T22:26:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:26:16.129 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 215 M 2024-05-08T22:26:16.130 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-05-08T22:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T22:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T22:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T22:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T22:26:16.131 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T22:26:16.135 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T22:26:16.137 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T22:26:16.371 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-05-08T22:26:16.386 INFO:teuthology.orchestra.run.smithi062.stdout:=================================================================================================================================== 2024-05-08T22:26:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-05-08T22:26:16.386 INFO:teuthology.orchestra.run.smithi062.stdout:=================================================================================================================================== 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T22:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T22:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-05-08T22:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T22:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T22:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T22:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T22:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T22:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T22:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T22:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T22:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T22:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T22:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T22:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T22:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T22:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T22:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T22:26:16.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T22:26:16.396 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T22:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling module streams: 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql 10 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout:=================================================================================================================================== 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout:Install 130 Packages 2024-05-08T22:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 16 Packages 2024-05-08T22:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 215 M 2024-05-08T22:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-05-08T22:26:16.492 INFO:teuthology.orchestra.run.smithi079.stdout:(17/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 14 MB/s | 6.5 MB 00:00 2024-05-08T22:26:16.551 INFO:teuthology.orchestra.run.smithi079.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 767 kB/s | 45 kB 00:00 2024-05-08T22:26:16.613 INFO:teuthology.orchestra.run.smithi062.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 35 kB/s | 6.4 kB 00:00 2024-05-08T22:26:16.616 INFO:teuthology.orchestra.run.smithi066.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 46 kB/s | 6.4 kB 00:00 2024-05-08T22:26:16.617 INFO:teuthology.orchestra.run.smithi079.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.9 MB/s | 129 kB 00:00 2024-05-08T22:26:16.676 INFO:teuthology.orchestra.run.smithi079.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 3.3 MB/s | 197 kB 00:00 2024-05-08T22:26:16.760 INFO:teuthology.orchestra.run.smithi079.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 4.4 MB/s | 370 kB 00:00 2024-05-08T22:26:16.852 INFO:teuthology.orchestra.run.smithi079.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 3.9 MB/s | 365 kB 00:00 2024-05-08T22:26:16.918 INFO:teuthology.orchestra.run.smithi079.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.6 MB/s | 107 kB 00:00 2024-05-08T22:26:16.977 INFO:teuthology.orchestra.run.smithi079.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.6 MB/s | 95 kB 00:00 2024-05-08T22:26:17.308 INFO:teuthology.orchestra.run.smithi066.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.2 MB/s | 860 kB 00:00 2024-05-08T22:26:17.310 INFO:teuthology.orchestra.run.smithi079.stdout:(25/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 10 MB/s | 3.5 MB 00:00 2024-05-08T22:26:17.369 INFO:teuthology.orchestra.run.smithi079.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 3.0 MB/s | 181 kB 00:00 2024-05-08T22:26:17.417 INFO:teuthology.orchestra.run.smithi066.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 1.5 MB/s | 166 kB 00:00 2024-05-08T22:26:17.419 INFO:teuthology.orchestra.run.smithi079.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 487 kB/s | 24 kB 00:00 2024-05-08T22:26:17.478 INFO:teuthology.orchestra.run.smithi079.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 2.3 MB/s | 138 kB 00:00 2024-05-08T22:26:17.521 INFO:teuthology.orchestra.run.smithi062.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 947 kB/s | 860 kB 00:00 2024-05-08T22:26:17.780 INFO:teuthology.orchestra.run.smithi062.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 643 kB/s | 166 kB 00:00 2024-05-08T22:26:17.795 INFO:teuthology.orchestra.run.smithi079.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 11 MB/s | 3.5 MB 00:00 2024-05-08T22:26:18.330 INFO:teuthology.orchestra.run.smithi062.stdout:(4/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 2.9 MB/s | 5.5 MB 00:01 2024-05-08T22:26:18.429 INFO:teuthology.orchestra.run.smithi079.stdout:(30/146): ceph-mgr-diskprediction-local-18.2.2- 12 MB/s | 7.4 MB 00:00 2024-05-08T22:26:18.495 INFO:teuthology.orchestra.run.smithi079.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 3.6 MB/s | 245 kB 00:00 2024-05-08T22:26:18.563 INFO:teuthology.orchestra.run.smithi079.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 737 kB/s | 49 kB 00:00 2024-05-08T22:26:18.621 INFO:teuthology.orchestra.run.smithi079.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 280 kB/s | 16 kB 00:00 2024-05-08T22:26:18.680 INFO:teuthology.orchestra.run.smithi062.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 4.5 MB/s | 1.6 MB 00:00 2024-05-08T22:26:18.697 INFO:teuthology.orchestra.run.smithi079.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 3.4 MB/s | 263 kB 00:00 2024-05-08T22:26:18.764 INFO:teuthology.orchestra.run.smithi079.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 2.5 MB/s | 170 kB 00:00 2024-05-08T22:26:18.830 INFO:teuthology.orchestra.run.smithi079.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 272 kB/s | 18 kB 00:00 2024-05-08T22:26:18.907 INFO:teuthology.orchestra.run.smithi066.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 1.5 MB/s | 2.3 MB 00:01 2024-05-08T22:26:19.289 INFO:teuthology.orchestra.run.smithi079.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 668 kB/s | 306 kB 00:00 2024-05-08T22:26:19.459 INFO:teuthology.orchestra.run.smithi066.stdout:(5/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 1.9 MB/s | 5.5 MB 00:02 2024-05-08T22:26:19.648 INFO:teuthology.orchestra.run.smithi079.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 164 kB/s | 59 kB 00:00 2024-05-08T22:26:19.772 INFO:teuthology.orchestra.run.smithi062.stdout:(6/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 4.4 MB/s | 4.8 MB 00:01 2024-05-08T22:26:19.793 INFO:teuthology.orchestra.run.smithi066.stdout:(6/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 1.8 MB/s | 1.6 MB 00:00 2024-05-08T22:26:19.831 INFO:teuthology.orchestra.run.smithi062.stdout:(7/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 1.1 MB/s | 2.3 MB 00:02 2024-05-08T22:26:19.957 INFO:teuthology.orchestra.run.smithi079.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 244 kB/s | 75 kB 00:00 2024-05-08T22:26:20.477 INFO:teuthology.orchestra.run.smithi079.stdout:(40/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 9.7 MB/s | 47 MB 00:04 2024-05-08T22:26:20.665 INFO:teuthology.orchestra.run.smithi079.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 671 kB/s | 476 kB 00:00 2024-05-08T22:26:20.807 INFO:teuthology.orchestra.run.smithi079.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 132 kB/s | 44 kB 00:00 2024-05-08T22:26:20.909 INFO:teuthology.orchestra.run.smithi066.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 3.3 MB/s | 4.8 MB 00:01 2024-05-08T22:26:21.049 INFO:teuthology.orchestra.run.smithi079.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 136 kB/s | 33 kB 00:00 2024-05-08T22:26:21.074 INFO:teuthology.orchestra.run.smithi079.stdout:(44/146): python3-beautifulsoup4-4.6.3-2.el8.1. 453 kB/s | 185 kB 00:00 2024-05-08T22:26:21.314 INFO:teuthology.orchestra.run.smithi079.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 62 kB/s | 16 kB 00:00 2024-05-08T22:26:21.483 INFO:teuthology.orchestra.run.smithi079.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 423 kB/s | 173 kB 00:00 2024-05-08T22:26:21.664 INFO:teuthology.orchestra.run.smithi062.stdout:(8/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el8 7.1 MB/s | 13 MB 00:01 2024-05-08T22:26:21.706 INFO:teuthology.orchestra.run.smithi062.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 587 kB/s | 24 kB 00:00 2024-05-08T22:26:21.800 INFO:teuthology.orchestra.run.smithi079.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 792 kB/s | 384 kB 00:00 2024-05-08T22:26:21.825 INFO:teuthology.orchestra.run.smithi079.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 238 kB/s | 81 kB 00:00 2024-05-08T22:26:22.042 INFO:teuthology.orchestra.run.smithi079.stdout:(49/146): ceph-common-18.2.2-1322.gd21e4f96.el8 2.3 MB/s | 22 MB 00:09 2024-05-08T22:26:22.159 INFO:teuthology.orchestra.run.smithi079.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 31 kB/s | 11 kB 00:00 2024-05-08T22:26:22.209 INFO:teuthology.orchestra.run.smithi079.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 46 kB/s | 18 kB 00:00 2024-05-08T22:26:22.468 INFO:teuthology.orchestra.run.smithi079.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 158 kB/s | 49 kB 00:00 2024-05-08T22:26:22.526 INFO:teuthology.orchestra.run.smithi079.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 185 kB/s | 59 kB 00:00 2024-05-08T22:26:22.777 INFO:teuthology.orchestra.run.smithi079.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 195 kB/s | 60 kB 00:00 2024-05-08T22:26:22.810 INFO:teuthology.orchestra.run.smithi079.stdout:(55/146): python3-kubernetes-11.0.0-6.el8.noarc 1.5 MB/s | 1.2 MB 00:00 2024-05-08T22:26:23.002 INFO:teuthology.orchestra.run.smithi079.stdout:(56/146): python3-pecan-1.3.2-9.el8.noarch.rpm 595 kB/s | 283 kB 00:00 2024-05-08T22:26:23.028 INFO:teuthology.orchestra.run.smithi079.stdout:(57/146): python3-portend-2.6-1.el8.noarch.rpm 62 kB/s | 16 kB 00:00 2024-05-08T22:26:23.053 INFO:teuthology.orchestra.run.smithi079.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 139 kB/s | 34 kB 00:00 2024-05-08T22:26:23.276 INFO:teuthology.orchestra.run.smithi066.stdout:(8/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el8 5.5 MB/s | 13 MB 00:02 2024-05-08T22:26:23.295 INFO:teuthology.orchestra.run.smithi079.stdout:(59/146): python3-simplegeneric-0.8.1-17.el8.no 77 kB/s | 19 kB 00:00 2024-05-08T22:26:23.318 INFO:teuthology.orchestra.run.smithi066.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 587 kB/s | 24 kB 00:00 2024-05-08T22:26:23.345 INFO:teuthology.orchestra.run.smithi079.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 202 kB/s | 64 kB 00:00 2024-05-08T22:26:23.395 INFO:teuthology.orchestra.run.smithi079.stdout:(61/146): python3-routes-2.4.1-12.el8.noarch.rp 500 kB/s | 196 kB 00:00 2024-05-08T22:26:23.546 INFO:teuthology.orchestra.run.smithi079.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 95 kB/s | 24 kB 00:00 2024-05-08T22:26:23.596 INFO:teuthology.orchestra.run.smithi079.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 117 kB/s | 29 kB 00:00 2024-05-08T22:26:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 114 kB/s | 27 kB 00:00 2024-05-08T22:26:23.710 INFO:teuthology.orchestra.run.smithi066.stdout:(10/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 4.7 MB/s | 18 MB 00:03 2024-05-08T22:26:23.793 INFO:teuthology.orchestra.run.smithi066.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 3.1 MB/s | 261 kB 00:00 2024-05-08T22:26:23.844 INFO:teuthology.orchestra.run.smithi066.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 620 kB/s | 31 kB 00:00 2024-05-08T22:26:23.980 INFO:teuthology.orchestra.run.smithi079.stdout:(65/146): python3-websocket-client-0.56.0-5.el8 179 kB/s | 61 kB 00:00 2024-05-08T22:26:24.035 INFO:teuthology.orchestra.run.smithi066.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 4.0 MB/s | 782 kB 00:00 2024-05-08T22:26:24.038 INFO:teuthology.orchestra.run.smithi079.stdout:(66/146): python3-waitress-1.4.3-1.el8.noarch.r 519 kB/s | 255 kB 00:00 2024-05-08T22:26:24.080 INFO:teuthology.orchestra.run.smithi079.stdout:(67/146): python3-webob-1.8.5-1.el8.1.noarch.rp 519 kB/s | 251 kB 00:00 2024-05-08T22:26:24.085 INFO:teuthology.orchestra.run.smithi066.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 2.8 MB/s | 142 kB 00:00 2024-05-08T22:26:24.144 INFO:teuthology.orchestra.run.smithi066.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 2.3 MB/s | 139 kB 00:00 2024-05-08T22:26:24.289 INFO:teuthology.orchestra.run.smithi079.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 92 kB/s | 23 kB 00:00 2024-05-08T22:26:24.294 INFO:teuthology.orchestra.run.smithi066.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 3.1 MB/s | 474 kB 00:00 2024-05-08T22:26:24.314 INFO:teuthology.orchestra.run.smithi079.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-05-08T22:26:24.340 INFO:teuthology.orchestra.run.smithi079.stdout:(70/146): python3-webtest-2.0.33-1.el8.noarch.r 237 kB/s | 85 kB 00:00 2024-05-08T22:26:24.498 INFO:teuthology.orchestra.run.smithi079.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 113 kB/s | 18 kB 00:00 2024-05-08T22:26:24.645 INFO:teuthology.orchestra.run.smithi079.stdout:(72/146): python3-jwt-1.6.1-2.el8.noarch.rpm 291 kB/s | 43 kB 00:00 2024-05-08T22:26:24.674 INFO:teuthology.orchestra.run.smithi079.stdout:(73/146): python3-packaging-16.8-9.el8.noarch.r 134 kB/s | 48 kB 00:00 2024-05-08T22:26:24.824 INFO:teuthology.orchestra.run.smithi079.stdout:(74/146): python3-oauthlib-2.1.0-1.el8.noarch.r 866 kB/s | 155 kB 00:00 2024-05-08T22:26:24.866 INFO:teuthology.orchestra.run.smithi079.stdout:(75/146): thrift-0.13.0-2.el8.x86_64.rpm 2.2 MB/s | 1.7 MB 00:00 2024-05-08T22:26:24.891 INFO:teuthology.orchestra.run.smithi079.stdout:(76/146): python3-pycparser-2.14-14.el8.noarch. 502 kB/s | 109 kB 00:00 2024-05-08T22:26:24.941 INFO:teuthology.orchestra.run.smithi079.stdout:(77/146): python3-requests-oauthlib-1.0.0-1.el8 364 kB/s | 43 kB 00:00 2024-05-08T22:26:24.992 INFO:teuthology.orchestra.run.smithi079.stdout:(78/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 149 kB/s | 19 kB 00:00 2024-05-08T22:26:25.017 INFO:teuthology.orchestra.run.smithi079.stdout:(79/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 365 kB/s | 44 kB 00:00 2024-05-08T22:26:25.142 INFO:teuthology.orchestra.run.smithi079.stdout:(80/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.9 MB/s | 605 kB 00:00 2024-05-08T22:26:25.176 INFO:teuthology.orchestra.run.smithi079.stdout:(81/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.2 MB/s | 229 kB 00:00 2024-05-08T22:26:25.201 INFO:teuthology.orchestra.run.smithi079.stdout:(82/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2024-05-08T22:26:25.293 INFO:teuthology.orchestra.run.smithi079.stdout:(83/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.3 MB/s | 200 kB 00:00 2024-05-08T22:26:25.318 INFO:teuthology.orchestra.run.smithi079.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 745 kB/s | 87 kB 00:00 2024-05-08T22:26:25.343 INFO:teuthology.orchestra.run.smithi079.stdout:(85/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 287 kB/s | 48 kB 00:00 2024-05-08T22:26:25.489 INFO:teuthology.orchestra.run.smithi079.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 226 kB/s | 39 kB 00:00 2024-05-08T22:26:25.519 INFO:teuthology.orchestra.run.smithi079.stdout:(87/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 255 kB/s | 58 kB 00:00 2024-05-08T22:26:25.544 INFO:teuthology.orchestra.run.smithi079.stdout:(88/146): fuse-2.9.7-17.el8.x86_64.rpm 414 kB/s | 83 kB 00:00 2024-05-08T22:26:25.595 INFO:teuthology.orchestra.run.smithi066.stdout:(17/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 5.0 MB/s | 6.5 MB 00:01 2024-05-08T22:26:25.620 INFO:teuthology.orchestra.run.smithi079.stdout:(89/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.9 MB/s | 645 kB 00:00 2024-05-08T22:26:25.645 INFO:teuthology.orchestra.run.smithi079.stdout:(90/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-05-08T22:26:25.645 INFO:teuthology.orchestra.run.smithi066.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 898 kB/s | 45 kB 00:00 2024-05-08T22:26:25.670 INFO:teuthology.orchestra.run.smithi079.stdout:(91/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.9 MB/s | 238 kB 00:00 2024-05-08T22:26:25.695 INFO:teuthology.orchestra.run.smithi066.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 2.5 MB/s | 129 kB 00:00 2024-05-08T22:26:25.754 INFO:teuthology.orchestra.run.smithi066.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 3.3 MB/s | 197 kB 00:00 2024-05-08T22:26:25.854 INFO:teuthology.orchestra.run.smithi066.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 3.6 MB/s | 370 kB 00:00 2024-05-08T22:26:25.946 INFO:teuthology.orchestra.run.smithi066.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 3.9 MB/s | 365 kB 00:00 2024-05-08T22:26:25.953 INFO:teuthology.orchestra.run.smithi079.stdout:(92/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 218 kB/s | 73 kB 00:00 2024-05-08T22:26:25.971 INFO:teuthology.orchestra.run.smithi079.stdout:(93/146): elfutils-libelf-devel-0.189-3.el8.x86 189 kB/s | 62 kB 00:00 2024-05-08T22:26:25.996 INFO:teuthology.orchestra.run.smithi066.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 2.1 MB/s | 107 kB 00:00 2024-05-08T22:26:26.004 INFO:teuthology.orchestra.run.smithi079.stdout:(94/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 1.6 MB/s | 562 kB 00:00 2024-05-08T22:26:26.076 INFO:teuthology.orchestra.run.smithi066.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.2 MB/s | 95 kB 00:00 2024-05-08T22:26:26.089 INFO:teuthology.orchestra.run.smithi062.stdout:(10/146): ceph-common-18.2.2-1322.gd21e4f96.el8 2.2 MB/s | 22 MB 00:09 2024-05-08T22:26:26.104 INFO:teuthology.orchestra.run.smithi079.stdout:(95/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 353 kB/s | 47 kB 00:00 2024-05-08T22:26:26.138 INFO:teuthology.orchestra.run.smithi079.stdout:(96/146): python3-cryptography-3.2.1-7.el8_9.x8 4.1 MB/s | 559 kB 00:00 2024-05-08T22:26:26.197 INFO:teuthology.orchestra.run.smithi079.stdout:(97/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 9.6 MB/s | 2.3 MB 00:00 2024-05-08T22:26:26.230 INFO:teuthology.orchestra.run.smithi079.stdout:(98/146): NetworkManager-initscripts-updown-1.4 1.1 MB/s | 144 kB 00:00 2024-05-08T22:26:26.264 INFO:teuthology.orchestra.run.smithi079.stdout:(99/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 795 kB/s | 100 kB 00:00 2024-05-08T22:26:26.265 INFO:teuthology.orchestra.run.smithi062.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 1.5 MB/s | 261 kB 00:00 2024-05-08T22:26:26.456 INFO:teuthology.orchestra.run.smithi079.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 77 kB/s | 20 kB 00:00 2024-05-08T22:26:26.473 INFO:teuthology.orchestra.run.smithi062.stdout:(12/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 2.7 MB/s | 18 MB 00:06 2024-05-08T22:26:26.490 INFO:teuthology.orchestra.run.smithi062.stdout:(13/146): libcephfs-devel-18.2.2-1322.gd21e4f96 137 kB/s | 31 kB 00:00 2024-05-08T22:26:26.539 INFO:teuthology.orchestra.run.smithi079.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 142 kB/s | 44 kB 00:00 2024-05-08T22:26:26.557 INFO:teuthology.orchestra.run.smithi062.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 2.1 MB/s | 142 kB 00:00 2024-05-08T22:26:26.565 INFO:teuthology.orchestra.run.smithi079.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 366 kB/s | 110 kB 00:00 2024-05-08T22:26:26.589 INFO:teuthology.orchestra.run.smithi066.stdout:(25/146): ceph-common-18.2.2-1322.gd21e4f96.el8 2.1 MB/s | 22 MB 00:10 2024-05-08T22:26:26.633 INFO:teuthology.orchestra.run.smithi062.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 1.8 MB/s | 139 kB 00:00 2024-05-08T22:26:26.714 INFO:teuthology.orchestra.run.smithi066.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.4 MB/s | 181 kB 00:00 2024-05-08T22:26:26.733 INFO:teuthology.orchestra.run.smithi062.stdout:(16/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 3.0 MB/s | 782 kB 00:00 2024-05-08T22:26:26.740 INFO:teuthology.orchestra.run.smithi079.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 93 kB/s | 27 kB 00:00 2024-05-08T22:26:26.774 INFO:teuthology.orchestra.run.smithi079.stdout:(104/146): python3-werkzeug-0.12.2-4.el8.noarch 2.1 MB/s | 457 kB 00:00 2024-05-08T22:26:26.790 INFO:teuthology.orchestra.run.smithi066.stdout:(27/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 4.8 MB/s | 3.5 MB 00:00 2024-05-08T22:26:26.807 INFO:teuthology.orchestra.run.smithi079.stdout:(105/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 473 kB/s | 126 kB 00:00 2024-05-08T22:26:26.808 INFO:teuthology.orchestra.run.smithi062.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.6 MB/s | 474 kB 00:00 2024-05-08T22:26:26.814 INFO:teuthology.orchestra.run.smithi066.stdout:(28/146): ceph-grafana-dashboards-18.2.2-1322.g 243 kB/s | 24 kB 00:00 2024-05-08T22:26:26.840 INFO:teuthology.orchestra.run.smithi066.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 2.7 MB/s | 138 kB 00:00 2024-05-08T22:26:26.866 INFO:teuthology.orchestra.run.smithi062.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 766 kB/s | 45 kB 00:00 2024-05-08T22:26:27.099 INFO:teuthology.orchestra.run.smithi079.stdout:(106/146): bison-3.0.4-10.el8.x86_64.rpm 1.9 MB/s | 688 kB 00:00 2024-05-08T22:26:27.158 INFO:teuthology.orchestra.run.smithi079.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 833 kB/s | 320 kB 00:00 2024-05-08T22:26:27.200 INFO:teuthology.orchestra.run.smithi079.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 1.8 MB/s | 709 kB 00:00 2024-05-08T22:26:27.225 INFO:teuthology.orchestra.run.smithi079.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 309 kB/s | 39 kB 00:00 2024-05-08T22:26:27.342 INFO:teuthology.orchestra.run.smithi062.stdout:(19/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 8.4 MB/s | 47 MB 00:05 2024-05-08T22:26:27.358 INFO:teuthology.orchestra.run.smithi079.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2024-05-08T22:26:27.367 INFO:teuthology.orchestra.run.smithi062.stdout:(20/146): python3-ceph-common-18.2.2-1322.gd21e 258 kB/s | 129 kB 00:00 2024-05-08T22:26:27.426 INFO:teuthology.orchestra.run.smithi062.stdout:(21/146): python3-cephfs-18.2.2-1322.gd21e4f96. 2.3 MB/s | 197 kB 00:00 2024-05-08T22:26:27.434 INFO:teuthology.orchestra.run.smithi079.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 274 kB/s | 64 kB 00:00 2024-05-08T22:26:27.526 INFO:teuthology.orchestra.run.smithi062.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 3.6 MB/s | 365 kB 00:00 2024-05-08T22:26:27.534 INFO:teuthology.orchestra.run.smithi079.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 335 kB/s | 103 kB 00:00 2024-05-08T22:26:27.560 INFO:teuthology.orchestra.run.smithi062.stdout:(23/146): python3-rados-18.2.2-1322.gd21e4f96.e 1.9 MB/s | 370 kB 00:00 2024-05-08T22:26:27.576 INFO:teuthology.orchestra.run.smithi062.stdout:(24/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 2.1 MB/s | 107 kB 00:00 2024-05-08T22:26:27.635 INFO:teuthology.orchestra.run.smithi062.stdout:(25/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.2 MB/s | 95 kB 00:00 2024-05-08T22:26:27.643 INFO:teuthology.orchestra.run.smithi079.stdout:(113/146): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2024-05-08T22:26:27.676 INFO:teuthology.orchestra.run.smithi079.stdout:(114/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.2 MB/s | 538 kB 00:00 2024-05-08T22:26:27.690 INFO:teuthology.orchestra.run.smithi066.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 4.0 MB/s | 3.5 MB 00:00 2024-05-08T22:26:27.752 INFO:teuthology.orchestra.run.smithi062.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.5 MB/s | 181 kB 00:00 2024-05-08T22:26:27.791 INFO:teuthology.orchestra.run.smithi066.stdout:(31/146): ceph-mgr-diskprediction-local-18.2.2- 7.8 MB/s | 7.4 MB 00:00 2024-05-08T22:26:27.793 INFO:teuthology.orchestra.run.smithi079.stdout:(115/146): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2024-05-08T22:26:27.819 INFO:teuthology.orchestra.run.smithi062.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 365 kB/s | 24 kB 00:00 2024-05-08T22:26:27.857 INFO:teuthology.orchestra.run.smithi066.stdout:(32/146): ceph-mgr-modules-core-18.2.2-1322.gd2 1.4 MB/s | 245 kB 00:00 2024-05-08T22:26:27.882 INFO:teuthology.orchestra.run.smithi066.stdout:(33/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 537 kB/s | 49 kB 00:00 2024-05-08T22:26:27.886 INFO:teuthology.orchestra.run.smithi062.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 2.0 MB/s | 138 kB 00:00 2024-05-08T22:26:27.916 INFO:teuthology.orchestra.run.smithi066.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1322.gd 278 kB/s | 16 kB 00:00 2024-05-08T22:26:27.966 INFO:teuthology.orchestra.run.smithi066.stdout:(35/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 3.1 MB/s | 263 kB 00:00 2024-05-08T22:26:27.970 INFO:teuthology.orchestra.run.smithi062.stdout:(29/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 8.8 MB/s | 3.5 MB 00:00 2024-05-08T22:26:27.985 INFO:teuthology.orchestra.run.smithi079.stdout:(116/146): socat-1.7.4.1-1.el8.x86_64.rpm 945 kB/s | 323 kB 00:00 2024-05-08T22:26:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:(36/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 2.2 MB/s | 170 kB 00:00 2024-05-08T22:26:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:(37/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 360 kB/s | 18 kB 00:00 2024-05-08T22:26:28.060 INFO:teuthology.orchestra.run.smithi079.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 586 kB/s | 225 kB 00:00 2024-05-08T22:26:28.094 INFO:teuthology.orchestra.run.smithi079.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.1 MB/s | 354 kB 00:00 2024-05-08T22:26:28.151 INFO:teuthology.orchestra.run.smithi066.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 441 kB/s | 59 kB 00:00 2024-05-08T22:26:28.209 INFO:teuthology.orchestra.run.smithi066.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2024-05-08T22:26:28.235 INFO:teuthology.orchestra.run.smithi066.stdout:(40/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2024-05-08T22:26:28.277 INFO:teuthology.orchestra.run.smithi066.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2024-05-08T22:26:28.327 INFO:teuthology.orchestra.run.smithi066.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2024-05-08T22:26:28.352 INFO:teuthology.orchestra.run.smithi066.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2024-05-08T22:26:28.369 INFO:teuthology.orchestra.run.smithi079.stdout:(119/146): openblas-threads-0.3.15-6.el8.x86_64 14 MB/s | 5.2 MB 00:00 2024-05-08T22:26:28.377 INFO:teuthology.orchestra.run.smithi066.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 659 kB/s | 33 kB 00:00 2024-05-08T22:26:28.394 INFO:teuthology.orchestra.run.smithi079.stdout:(120/146): python3-mako-1.0.6-14.el8.noarch.rpm 470 kB/s | 157 kB 00:00 2024-05-08T22:26:28.403 INFO:teuthology.orchestra.run.smithi066.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2024-05-08T22:26:28.445 INFO:teuthology.orchestra.run.smithi066.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2024-05-08T22:26:28.487 INFO:teuthology.orchestra.run.smithi066.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2024-05-08T22:26:28.520 INFO:teuthology.orchestra.run.smithi079.stdout:(121/146): openblas-0.3.15-6.el8.x86_64.rpm 12 MB/s | 5.0 MB 00:00 2024-05-08T22:26:28.520 INFO:teuthology.orchestra.run.smithi066.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 335 kB/s | 11 kB 00:00 2024-05-08T22:26:28.536 INFO:teuthology.orchestra.run.smithi062.stdout:(30/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 3.6 MB/s | 6.5 MB 00:01 2024-05-08T22:26:28.554 INFO:teuthology.orchestra.run.smithi066.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 529 kB/s | 18 kB 00:00 2024-05-08T22:26:28.579 INFO:teuthology.orchestra.run.smithi066.stdout:(50/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2024-05-08T22:26:28.603 INFO:teuthology.orchestra.run.smithi079.stdout:(122/146): libpq-13.11-1.el8.x86_64.rpm 949 kB/s | 198 kB 00:00 2024-05-08T22:26:28.612 INFO:teuthology.orchestra.run.smithi066.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2024-05-08T22:26:28.637 INFO:teuthology.orchestra.run.smithi079.stdout:(123/146): automake-1.16.1-8.el8.noarch.rpm 2.6 MB/s | 714 kB 00:00 2024-05-08T22:26:28.646 INFO:teuthology.orchestra.run.smithi066.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 1.7 MB/s | 59 kB 00:00 2024-05-08T22:26:28.653 INFO:teuthology.orchestra.run.smithi062.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 2.1 MB/s | 245 kB 00:00 2024-05-08T22:26:28.680 INFO:teuthology.orchestra.run.smithi066.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2024-05-08T22:26:28.745 INFO:teuthology.orchestra.run.smithi062.stdout:(32/146): ceph-mgr-diskprediction-local-18.2.2- 9.6 MB/s | 7.4 MB 00:00 2024-05-08T22:26:28.771 INFO:teuthology.orchestra.run.smithi062.stdout:(33/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 421 kB/s | 49 kB 00:00 2024-05-08T22:26:28.772 INFO:teuthology.orchestra.run.smithi066.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 3.0 MB/s | 283 kB 00:00 2024-05-08T22:26:28.796 INFO:teuthology.orchestra.run.smithi062.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1322.gd 325 kB/s | 16 kB 00:00 2024-05-08T22:26:28.806 INFO:teuthology.orchestra.run.smithi066.stdout:(55/146): python3-kubernetes-11.0.0-6.el8.noarc 4.6 MB/s | 1.2 MB 00:00 2024-05-08T22:26:28.817 INFO:teuthology.orchestra.run.smithi079.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 682 kB/s | 202 kB 00:00 2024-05-08T22:26:28.823 INFO:teuthology.orchestra.run.smithi066.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2024-05-08T22:26:28.839 INFO:teuthology.orchestra.run.smithi066.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2024-05-08T22:26:28.846 INFO:teuthology.orchestra.run.smithi079.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 224 kB/s | 54 kB 00:00 2024-05-08T22:26:28.863 INFO:teuthology.orchestra.run.smithi062.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 2.5 MB/s | 170 kB 00:00 2024-05-08T22:26:28.882 INFO:teuthology.orchestra.run.smithi066.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2024-05-08T22:26:28.907 INFO:teuthology.orchestra.run.smithi066.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 960 kB/s | 64 kB 00:00 2024-05-08T22:26:28.921 INFO:teuthology.orchestra.run.smithi062.stdout:(36/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 3.4 MB/s | 3.5 MB 00:01 2024-05-08T22:26:28.932 INFO:teuthology.orchestra.run.smithi066.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2024-05-08T22:26:28.957 INFO:teuthology.orchestra.run.smithi066.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2024-05-08T22:26:28.971 INFO:teuthology.orchestra.run.smithi079.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-08T22:26:28.983 INFO:teuthology.orchestra.run.smithi066.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 583 kB/s | 29 kB 00:00 2024-05-08T22:26:29.013 INFO:teuthology.orchestra.run.smithi062.stdout:(37/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 1.1 MB/s | 263 kB 00:00 2024-05-08T22:26:29.025 INFO:teuthology.orchestra.run.smithi066.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 412 kB/s | 27 kB 00:00 2024-05-08T22:26:29.038 INFO:teuthology.orchestra.run.smithi079.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 7.8 MB/s | 1.5 MB 00:00 2024-05-08T22:26:29.067 INFO:teuthology.orchestra.run.smithi066.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2024-05-08T22:26:29.109 INFO:teuthology.orchestra.run.smithi066.stdout:(65/146): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2024-05-08T22:26:29.151 INFO:teuthology.orchestra.run.smithi066.stdout:(66/146): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2024-05-08T22:26:29.193 INFO:teuthology.orchestra.run.smithi066.stdout:(67/146): python3-zc-lockfile-2.0-2.el8.noarch. 551 kB/s | 23 kB 00:00 2024-05-08T22:26:29.213 INFO:teuthology.orchestra.run.smithi079.stdout:(128/146): python3-scipy-1.0.0-21.module+el8.9. 24 MB/s | 14 MB 00:00 2024-05-08T22:26:29.247 INFO:teuthology.orchestra.run.smithi079.stdout:(129/146): postgresql-contrib-10.23-4.module+el 2.9 MB/s | 812 kB 00:00 2024-05-08T22:26:29.364 INFO:teuthology.orchestra.run.smithi062.stdout:(38/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 36 kB/s | 18 kB 00:00 2024-05-08T22:26:29.376 INFO:teuthology.orchestra.run.smithi066.stdout:(68/146): python3-webob-1.8.5-1.el8.1.noarch.rp 716 kB/s | 251 kB 00:00 2024-05-08T22:26:29.401 INFO:teuthology.orchestra.run.smithi066.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-05-08T22:26:29.481 INFO:teuthology.orchestra.run.smithi062.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 126 kB/s | 59 kB 00:00 2024-05-08T22:26:29.514 INFO:teuthology.orchestra.run.smithi062.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 500 kB/s | 75 kB 00:00 2024-05-08T22:26:29.526 INFO:teuthology.orchestra.run.smithi066.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 5.2 MB/s | 1.7 MB 00:00 2024-05-08T22:26:29.556 INFO:teuthology.orchestra.run.smithi062.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2024-05-08T22:26:29.564 INFO:teuthology.orchestra.run.smithi079.stdout:(130/146): postgresql-server-10.23-4.module+el8 9.6 MB/s | 5.1 MB 00:00 2024-05-08T22:26:29.573 INFO:teuthology.orchestra.run.smithi062.stdout:(42/146): gperftools-libs-2.7-9.el8.x86_64.rpm 547 kB/s | 306 kB 00:00 2024-05-08T22:26:29.585 INFO:teuthology.orchestra.run.smithi066.stdout:(71/146): python3-packaging-16.8-9.el8.noarch.r 263 kB/s | 48 kB 00:00 2024-05-08T22:26:29.598 INFO:teuthology.orchestra.run.smithi062.stdout:(43/146): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2024-05-08T22:26:29.616 INFO:teuthology.orchestra.run.smithi062.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 788 kB/s | 33 kB 00:00 2024-05-08T22:26:29.633 INFO:teuthology.orchestra.run.smithi062.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 486 kB/s | 16 kB 00:00 2024-05-08T22:26:29.656 INFO:teuthology.orchestra.run.smithi079.stdout:(131/146): librados2-18.2.2-1322.gd21e4f96.el8. 8.4 MB/s | 3.7 MB 00:00 2024-05-08T22:26:29.690 INFO:teuthology.orchestra.run.smithi062.stdout:(46/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2024-05-08T22:26:29.725 INFO:teuthology.orchestra.run.smithi062.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.5 MB/s | 173 kB 00:00 2024-05-08T22:26:29.731 INFO:teuthology.orchestra.run.smithi079.stdout:(132/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 7.4 MB/s | 3.6 MB 00:00 2024-05-08T22:26:29.741 INFO:teuthology.orchestra.run.smithi062.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.5 MB/s | 81 kB 00:00 2024-05-08T22:26:29.756 INFO:teuthology.orchestra.run.smithi079.stdout:(133/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.2 MB/s | 232 kB 00:00 2024-05-08T22:26:29.758 INFO:teuthology.orchestra.run.smithi062.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 334 kB/s | 11 kB 00:00 2024-05-08T22:26:29.773 INFO:teuthology.orchestra.run.smithi079.stdout:(134/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.5 MB/s | 303 kB 00:00 2024-05-08T22:26:29.784 INFO:teuthology.orchestra.run.smithi062.stdout:(50/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2024-05-08T22:26:29.801 INFO:teuthology.orchestra.run.smithi062.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 299 kB/s | 18 kB 00:00 2024-05-08T22:26:29.826 INFO:teuthology.orchestra.run.smithi062.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2024-05-08T22:26:29.843 INFO:teuthology.orchestra.run.smithi062.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2024-05-08T22:26:29.860 INFO:teuthology.orchestra.run.smithi062.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2024-05-08T22:26:29.882 INFO:teuthology.orchestra.run.smithi079.stdout:(135/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.5 MB/s | 854 kB 00:00 2024-05-08T22:26:29.902 INFO:teuthology.orchestra.run.smithi062.stdout:(55/146): python3-kubernetes-11.0.0-6.el8.noarc 8.2 MB/s | 1.2 MB 00:00 2024-05-08T22:26:29.915 INFO:teuthology.orchestra.run.smithi079.stdout:(136/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.4 MB/s | 711 kB 00:00 2024-05-08T22:26:29.919 INFO:teuthology.orchestra.run.smithi062.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2024-05-08T22:26:29.944 INFO:teuthology.orchestra.run.smithi062.stdout:(57/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2024-05-08T22:26:29.961 INFO:teuthology.orchestra.run.smithi062.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 574 kB/s | 34 kB 00:00 2024-05-08T22:26:29.974 INFO:teuthology.orchestra.run.smithi079.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 3.6 MB/s | 333 kB 00:00 2024-05-08T22:26:29.986 INFO:teuthology.orchestra.run.smithi062.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2024-05-08T22:26:30.012 INFO:teuthology.orchestra.run.smithi062.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 374 kB/s | 19 kB 00:00 2024-05-08T22:26:30.024 INFO:teuthology.orchestra.run.smithi079.stdout:(138/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 5.9 MB/s | 1.5 MB 00:00 2024-05-08T22:26:30.027 INFO:teuthology.orchestra.run.smithi066.stdout:(72/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 7.0 MB/s | 47 MB 00:06 2024-05-08T22:26:30.037 INFO:teuthology.orchestra.run.smithi062.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 478 kB/s | 24 kB 00:00 2024-05-08T22:26:30.049 INFO:teuthology.orchestra.run.smithi079.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 2.9 MB/s | 221 kB 00:00 2024-05-08T22:26:30.052 INFO:teuthology.orchestra.run.smithi066.stdout:(73/146): python3-jsonpointer-1.10-11.el8.noarc 38 kB/s | 20 kB 00:00 2024-05-08T22:26:30.062 INFO:teuthology.orchestra.run.smithi062.stdout:(62/146): python3-rsa-4.9-2.el8.noarch.rpm 546 kB/s | 64 kB 00:00 2024-05-08T22:26:30.083 INFO:teuthology.orchestra.run.smithi079.stdout:(140/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.4 MB/s | 1.1 MB 00:00 2024-05-08T22:26:30.086 INFO:teuthology.orchestra.run.smithi066.stdout:(74/146): python3-prettytable-0.7.2-14.el8.noar 88 kB/s | 44 kB 00:00 2024-05-08T22:26:30.087 INFO:teuthology.orchestra.run.smithi062.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2024-05-08T22:26:30.113 INFO:teuthology.orchestra.run.smithi062.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2024-05-08T22:26:30.138 INFO:teuthology.orchestra.run.smithi062.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2024-05-08T22:26:30.142 INFO:teuthology.orchestra.run.smithi079.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.1 MB/s | 254 kB 00:00 2024-05-08T22:26:30.155 INFO:teuthology.orchestra.run.smithi062.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2024-05-08T22:26:30.180 INFO:teuthology.orchestra.run.smithi062.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 915 kB/s | 61 kB 00:00 2024-05-08T22:26:30.206 INFO:teuthology.orchestra.run.smithi062.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 458 kB/s | 23 kB 00:00 2024-05-08T22:26:30.217 INFO:teuthology.orchestra.run.smithi079.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.4 MB/s | 237 kB 00:00 2024-05-08T22:26:30.231 INFO:teuthology.orchestra.run.smithi062.stdout:(69/146): python3-webtest-2.0.33-1.el8.noarch.r 924 kB/s | 85 kB 00:00 2024-05-08T22:26:30.242 INFO:teuthology.orchestra.run.smithi079.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.1 MB/s | 180 kB 00:00 2024-05-08T22:26:30.247 INFO:teuthology.orchestra.run.smithi062.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2024-05-08T22:26:30.259 INFO:teuthology.orchestra.run.smithi079.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 846 kB/s | 99 kB 00:00 2024-05-08T22:26:30.303 INFO:teuthology.orchestra.run.smithi066.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 110 kB/s | 27 kB 00:00 2024-05-08T22:26:30.306 INFO:teuthology.orchestra.run.smithi062.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2024-05-08T22:26:30.320 INFO:teuthology.orchestra.run.smithi066.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 541 kB/s | 126 kB 00:00 2024-05-08T22:26:30.345 INFO:teuthology.orchestra.run.smithi066.stdout:(77/146): python3-pyasn1-modules-0.3.7-6.el8.no 347 kB/s | 110 kB 00:00 2024-05-08T22:26:30.373 INFO:teuthology.orchestra.run.smithi062.stdout:(72/146): python3-packaging-16.8-9.el8.noarch.r 339 kB/s | 48 kB 00:00 2024-05-08T22:26:30.376 INFO:teuthology.orchestra.run.smithi079.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 16 MB/s | 2.5 MB 00:00 2024-05-08T22:26:30.401 INFO:teuthology.orchestra.run.smithi079.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 648 kB/s | 103 kB 00:00 2024-05-08T22:26:30.404 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:26:30.404 INFO:teuthology.orchestra.run.smithi079.stdout:Total 12 MB/s | 215 MB 00:17 2024-05-08T22:26:30.432 INFO:teuthology.orchestra.run.smithi062.stdout:(73/146): python3-jsonpointer-1.10-11.el8.noarc 114 kB/s | 20 kB 00:00 2024-05-08T22:26:30.454 INFO:teuthology.orchestra.run.smithi066.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 3.0 MB/s | 457 kB 00:00 2024-05-08T22:26:30.524 INFO:teuthology.orchestra.run.smithi062.stdout:(74/146): python3-prettytable-0.7.2-14.el8.noar 210 kB/s | 44 kB 00:00 2024-05-08T22:26:30.546 INFO:teuthology.orchestra.run.smithi066.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2024-05-08T22:26:30.567 INFO:teuthology.orchestra.run.smithi062.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 197 kB/s | 27 kB 00:00 2024-05-08T22:26:30.579 INFO:teuthology.orchestra.run.smithi066.stdout:(80/146): bison-3.0.4-10.el8.x86_64.rpm 2.6 MB/s | 688 kB 00:00 2024-05-08T22:26:30.599 INFO:teuthology.orchestra.run.smithi062.stdout:(76/146): python3-pyasn1-modules-0.3.7-6.el8.no 487 kB/s | 110 kB 00:00 2024-05-08T22:26:30.613 INFO:teuthology.orchestra.run.smithi066.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 4.4 MB/s | 709 kB 00:00 2024-05-08T22:26:30.633 INFO:teuthology.orchestra.run.smithi062.stdout:(77/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-05-08T22:26:30.638 INFO:teuthology.orchestra.run.smithi066.stdout:(82/146): python3-markupsafe-0.23-19.el8.x86_64 421 kB/s | 39 kB 00:00 2024-05-08T22:26:30.705 INFO:teuthology.orchestra.run.smithi066.stdout:(83/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2024-05-08T22:26:30.725 INFO:teuthology.orchestra.run.smithi062.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.8 MB/s | 457 kB 00:00 2024-05-08T22:26:30.730 INFO:teuthology.orchestra.run.smithi066.stdout:(84/146): uuid-1.6.2-43.el8.x86_64.rpm 546 kB/s | 64 kB 00:00 2024-05-08T22:26:30.759 INFO:teuthology.orchestra.run.smithi062.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 2.5 MB/s | 320 kB 00:00 2024-05-08T22:26:30.764 INFO:teuthology.orchestra.run.smithi066.stdout:(85/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 824 kB/s | 103 kB 00:00 2024-05-08T22:26:30.784 INFO:teuthology.orchestra.run.smithi062.stdout:(80/146): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2024-05-08T22:26:30.831 INFO:teuthology.orchestra.run.smithi066.stdout:(86/146): python3-jinja2-2.10.1-3.el8.noarch.rp 5.2 MB/s | 538 kB 00:00 2024-05-08T22:26:30.851 INFO:teuthology.orchestra.run.smithi062.stdout:(81/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2024-05-08T22:26:30.901 INFO:teuthology.orchestra.run.smithi062.stdout:(82/146): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2024-05-08T22:26:30.923 INFO:teuthology.orchestra.run.smithi066.stdout:(87/146): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2024-05-08T22:26:30.926 INFO:teuthology.orchestra.run.smithi062.stdout:(83/146): python3-markupsafe-0.23-19.el8.x86_64 232 kB/s | 39 kB 00:00 2024-05-08T22:26:30.952 INFO:teuthology.orchestra.run.smithi062.stdout:(84/146): uuid-1.6.2-43.el8.x86_64.rpm 637 kB/s | 64 kB 00:00 2024-05-08T22:26:30.956 INFO:teuthology.orchestra.run.smithi066.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.5 MB/s | 323 kB 00:00 2024-05-08T22:26:31.002 INFO:teuthology.orchestra.run.smithi062.stdout:(85/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.0 MB/s | 103 kB 00:00 2024-05-08T22:26:31.040 INFO:teuthology.orchestra.run.smithi066.stdout:(89/146): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2024-05-08T22:26:31.065 INFO:teuthology.orchestra.run.smithi066.stdout:(90/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2024-05-08T22:26:31.099 INFO:teuthology.orchestra.run.smithi066.stdout:(91/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2024-05-08T22:26:31.136 INFO:teuthology.orchestra.run.smithi062.stdout:(86/146): python3-numpy-1.14.3-10.el8.x86_64.rp 28 MB/s | 3.7 MB 00:00 2024-05-08T22:26:31.158 INFO:teuthology.orchestra.run.smithi066.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2024-05-08T22:26:31.169 INFO:teuthology.orchestra.run.smithi062.stdout:(87/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.4 MB/s | 538 kB 00:00 2024-05-08T22:26:31.250 INFO:teuthology.orchestra.run.smithi066.stdout:(93/146): openblas-threads-0.3.15-6.el8.x86_64. 25 MB/s | 5.2 MB 00:00 2024-05-08T22:26:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:(88/146): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2024-05-08T22:26:31.292 INFO:teuthology.orchestra.run.smithi066.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 5.2 MB/s | 714 kB 00:00 2024-05-08T22:26:31.295 INFO:teuthology.orchestra.run.smithi062.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.8 MB/s | 225 kB 00:00 2024-05-08T22:26:31.320 INFO:teuthology.orchestra.run.smithi062.stdout:(90/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2024-05-08T22:26:31.400 INFO:teuthology.orchestra.run.smithi066.stdout:(95/146): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-05-08T22:26:31.411 INFO:teuthology.orchestra.run.smithi062.stdout:(91/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.5 MB/s | 354 kB 00:00 2024-05-08T22:26:31.450 INFO:teuthology.orchestra.run.smithi066.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 988 kB/s | 198 kB 00:00 2024-05-08T22:26:31.466 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-05-08T22:26:31.475 INFO:teuthology.orchestra.run.smithi066.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2024-05-08T22:26:31.529 INFO:teuthology.orchestra.run.smithi062.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 22 MB/s | 5.2 MB 00:00 2024-05-08T22:26:31.542 INFO:teuthology.orchestra.run.smithi066.stdout:(98/146): python3-pytz-2017.2-11.el8.noarch.rpm 381 kB/s | 54 kB 00:00 2024-05-08T22:26:31.554 INFO:teuthology.orchestra.run.smithi062.stdout:(93/146): python3-mako-1.0.6-14.el8.noarch.rpm 671 kB/s | 157 kB 00:00 2024-05-08T22:26:31.584 INFO:teuthology.orchestra.run.smithi066.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-08T22:26:31.663 INFO:teuthology.orchestra.run.smithi062.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 20 MB/s | 5.0 MB 00:00 2024-05-08T22:26:31.704 INFO:teuthology.orchestra.run.smithi062.stdout:(95/146): automake-1.16.1-8.el8.noarch.rpm 4.0 MB/s | 714 kB 00:00 2024-05-08T22:26:31.718 INFO:teuthology.orchestra.run.smithi066.stdout:(100/146): postgresql-contrib-10.23-4.module+el 6.0 MB/s | 812 kB 00:00 2024-05-08T22:26:31.730 INFO:teuthology.orchestra.run.smithi062.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2024-05-08T22:26:31.777 INFO:teuthology.orchestra.run.smithi066.stdout:(101/146): postgresql-10.23-4.module+el8.9.0+21 6.4 MB/s | 1.5 MB 00:00 2024-05-08T22:26:31.797 INFO:teuthology.orchestra.run.smithi062.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 589 kB/s | 54 kB 00:00 2024-05-08T22:26:31.822 INFO:teuthology.orchestra.run.smithi062.stdout:(98/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-05-08T22:26:31.858 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-05-08T22:26:31.858 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-05-08T22:26:31.889 INFO:teuthology.orchestra.run.smithi062.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-08T22:26:31.944 INFO:teuthology.orchestra.run.smithi066.stdout:(102/146): python3-scipy-1.0.0-21.module+el8.9. 28 MB/s | 14 MB 00:00 2024-05-08T22:26:31.969 INFO:teuthology.orchestra.run.smithi066.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 93 kB/s | 18 kB 00:00 2024-05-08T22:26:31.981 INFO:teuthology.orchestra.run.smithi062.stdout:(100/146): postgresql-10.23-4.module+el8.9.0+21 9.5 MB/s | 1.5 MB 00:00 2024-05-08T22:26:32.031 INFO:teuthology.orchestra.run.smithi062.stdout:(101/146): postgresql-contrib-10.23-4.module+el 5.6 MB/s | 812 kB 00:00 2024-05-08T22:26:32.094 INFO:teuthology.orchestra.run.smithi066.stdout:(104/146): postgresql-server-10.23-4.module+el8 13 MB/s | 5.1 MB 00:00 2024-05-08T22:26:32.119 INFO:teuthology.orchestra.run.smithi066.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 243 kB/s | 43 kB 00:00 2024-05-08T22:26:32.162 INFO:teuthology.orchestra.run.smithi066.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 805 kB/s | 155 kB 00:00 2024-05-08T22:26:32.165 INFO:teuthology.orchestra.run.smithi062.stdout:(102/146): libverto-devel-0.3.0-5.el8.x86_64.rp 134 kB/s | 18 kB 00:00 2024-05-08T22:26:32.195 INFO:teuthology.orchestra.run.smithi066.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 1.1 MB/s | 109 kB 00:00 2024-05-08T22:26:32.228 INFO:teuthology.orchestra.run.smithi066.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 392 kB/s | 43 kB 00:00 2024-05-08T22:26:32.240 INFO:teuthology.orchestra.run.smithi062.stdout:(103/146): postgresql-server-10.23-4.module+el8 19 MB/s | 5.1 MB 00:00 2024-05-08T22:26:32.362 INFO:teuthology.orchestra.run.smithi066.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 93 kB/s | 19 kB 00:00 2024-05-08T22:26:32.387 INFO:teuthology.orchestra.run.smithi066.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 228 kB/s | 44 kB 00:00 2024-05-08T22:26:32.399 INFO:teuthology.orchestra.run.smithi062.stdout:(104/146): python3-scipy-1.0.0-21.module+el8.9. 20 MB/s | 14 MB 00:00 2024-05-08T22:26:32.424 INFO:teuthology.orchestra.run.smithi062.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 165 kB/s | 43 kB 00:00 2024-05-08T22:26:32.463 INFO:teuthology.orchestra.run.smithi066.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.5 MB/s | 605 kB 00:00 2024-05-08T22:26:32.474 INFO:teuthology.orchestra.run.smithi062.stdout:(106/146): python3-requests-oauthlib-1.0.0-1.el 854 kB/s | 43 kB 00:00 2024-05-08T22:26:32.500 INFO:teuthology.orchestra.run.smithi062.stdout:(107/146): python3-oauthlib-2.1.0-1.el8.noarch. 598 kB/s | 155 kB 00:00 2024-05-08T22:26:32.513 INFO:teuthology.orchestra.run.smithi066.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2024-05-08T22:26:32.525 INFO:teuthology.orchestra.run.smithi062.stdout:(108/146): python3-pycparser-2.14-14.el8.noarch 868 kB/s | 109 kB 00:00 2024-05-08T22:26:32.538 INFO:teuthology.orchestra.run.smithi066.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-05-08T22:26:32.563 INFO:teuthology.orchestra.run.smithi066.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.9 MB/s | 200 kB 00:00 2024-05-08T22:26:32.567 INFO:teuthology.orchestra.run.smithi062.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 203 kB/s | 19 kB 00:00 2024-05-08T22:26:32.601 INFO:teuthology.orchestra.run.smithi062.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 436 kB/s | 44 kB 00:00 2024-05-08T22:26:32.626 INFO:teuthology.orchestra.run.smithi062.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 5.9 MB/s | 605 kB 00:00 2024-05-08T22:26:32.639 INFO:teuthology.orchestra.run.smithi066.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 382 kB/s | 48 kB 00:00 2024-05-08T22:26:32.685 INFO:teuthology.orchestra.run.smithi062.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.9 MB/s | 229 kB 00:00 2024-05-08T22:26:32.698 INFO:teuthology.orchestra.run.smithi066.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 431 kB/s | 58 kB 00:00 2024-05-08T22:26:32.709 INFO:teuthology.orchestra.run.smithi062.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.0 MB/s | 220 kB 00:00 2024-05-08T22:26:32.714 INFO:teuthology.orchestra.run.smithi066.stdout:(117/146): libsepol-devel-2.9-3.el8.x86_64.rpm 494 kB/s | 87 kB 00:00 2024-05-08T22:26:32.760 INFO:teuthology.orchestra.run.smithi062.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.5 MB/s | 200 kB 00:00 2024-05-08T22:26:32.764 INFO:teuthology.orchestra.run.smithi066.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 307 kB/s | 39 kB 00:00 2024-05-08T22:26:32.798 INFO:teuthology.orchestra.run.smithi066.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 828 kB/s | 83 kB 00:00 2024-05-08T22:26:32.810 INFO:teuthology.orchestra.run.smithi062.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 384 kB/s | 48 kB 00:00 2024-05-08T22:26:32.827 INFO:teuthology.orchestra.run.smithi062.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 741 kB/s | 87 kB 00:00 2024-05-08T22:26:32.840 INFO:teuthology.orchestra.run.smithi066.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 5.0 MB/s | 645 kB 00:00 2024-05-08T22:26:32.890 INFO:teuthology.orchestra.run.smithi066.stdout:(121/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2024-05-08T22:26:32.894 INFO:teuthology.orchestra.run.smithi062.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 431 kB/s | 58 kB 00:00 2024-05-08T22:26:32.919 INFO:teuthology.orchestra.run.smithi062.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 354 kB/s | 39 kB 00:00 2024-05-08T22:26:32.945 INFO:teuthology.orchestra.run.smithi062.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 709 kB/s | 83 kB 00:00 2024-05-08T22:26:32.957 INFO:teuthology.orchestra.run.smithi066.stdout:(122/146): libquadmath-8.5.0-20.el8.x86_64.rpm 895 kB/s | 172 kB 00:00 2024-05-08T22:26:33.024 INFO:teuthology.orchestra.run.smithi066.stdout:(123/146): elfutils-libelf-devel-0.189-3.el8.x8 462 kB/s | 62 kB 00:00 2024-05-08T22:26:33.037 INFO:teuthology.orchestra.run.smithi062.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.4 MB/s | 172 kB 00:00 2024-05-08T22:26:33.050 INFO:teuthology.orchestra.run.smithi066.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 347 kB/s | 73 kB 00:00 2024-05-08T22:26:33.072 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-05-08T22:26:33.082 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-05-08T22:26:33.087 INFO:teuthology.orchestra.run.smithi062.stdout:(121/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.3 MB/s | 645 kB 00:00 2024-05-08T22:26:33.116 INFO:teuthology.orchestra.run.smithi066.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.5 MB/s | 562 kB 00:00 2024-05-08T22:26:33.154 INFO:teuthology.orchestra.run.smithi062.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.1 MB/s | 238 kB 00:00 2024-05-08T22:26:33.192 INFO:teuthology.orchestra.run.smithi066.stdout:(126/146): openssl-devel-1.1.1k-12.el8_9.x86_64 14 MB/s | 2.3 MB 00:00 2024-05-08T22:26:33.198 INFO:teuthology.orchestra.run.smithi062.stdout:(123/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 457 kB/s | 73 kB 00:00 2024-05-08T22:26:33.221 INFO:teuthology.orchestra.run.smithi062.stdout:(124/146): elfutils-libelf-devel-0.189-3.el8.x8 461 kB/s | 62 kB 00:00 2024-05-08T22:26:33.242 INFO:teuthology.orchestra.run.smithi066.stdout:(127/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 245 kB/s | 47 kB 00:00 2024-05-08T22:26:33.271 INFO:teuthology.orchestra.run.smithi062.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 4.7 MB/s | 562 kB 00:00 2024-05-08T22:26:33.275 INFO:teuthology.orchestra.run.smithi066.stdout:(128/146): python3-cryptography-3.2.1-7.el8_9.x 3.4 MB/s | 559 kB 00:00 2024-05-08T22:26:33.343 INFO:teuthology.orchestra.run.smithi066.stdout:(129/146): NetworkManager-initscripts-updown-1. 960 kB/s | 144 kB 00:00 2024-05-08T22:26:33.355 INFO:teuthology.orchestra.run.smithi062.stdout:(126/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 353 kB/s | 47 kB 00:00 2024-05-08T22:26:33.368 INFO:teuthology.orchestra.run.smithi066.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 796 kB/s | 100 kB 00:00 2024-05-08T22:26:33.405 INFO:teuthology.orchestra.run.smithi062.stdout:(127/146): python3-cryptography-3.2.1-7.el8_9.x 4.1 MB/s | 559 kB 00:00 2024-05-08T22:26:33.472 INFO:teuthology.orchestra.run.smithi062.stdout:(128/146): openssl-devel-1.1.1k-12.el8_9.x86_64 8.4 MB/s | 2.3 MB 00:00 2024-05-08T22:26:33.497 INFO:teuthology.orchestra.run.smithi062.stdout:(129/146): NetworkManager-initscripts-updown-1. 1.0 MB/s | 144 kB 00:00 2024-05-08T22:26:33.501 INFO:teuthology.orchestra.run.smithi066.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.7 MB/s | 232 kB 00:00 2024-05-08T22:26:33.522 INFO:teuthology.orchestra.run.smithi062.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 853 kB/s | 100 kB 00:00 2024-05-08T22:26:33.627 INFO:teuthology.orchestra.run.smithi066.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.4 MB/s | 303 kB 00:00 2024-05-08T22:26:33.631 INFO:teuthology.orchestra.run.smithi062.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 2.1 MB/s | 232 kB 00:00 2024-05-08T22:26:33.748 INFO:teuthology.orchestra.run.smithi062.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.5 MB/s | 303 kB 00:00 2024-05-08T22:26:33.810 INFO:teuthology.orchestra.run.smithi066.stdout:(133/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 7.6 MB/s | 3.6 MB 00:00 2024-05-08T22:26:33.860 INFO:teuthology.orchestra.run.smithi066.stdout:(134/146): librados2-18.2.2-1322.gd21e4f96.el8. 6.3 MB/s | 3.7 MB 00:00 2024-05-08T22:26:33.902 INFO:teuthology.orchestra.run.smithi066.stdout:(135/146): iproute-6.2.0-5.el8_9.x86_64.rpm 3.0 MB/s | 854 kB 00:00 2024-05-08T22:26:33.924 INFO:teuthology.orchestra.run.smithi062.stdout:(133/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 8.4 MB/s | 3.6 MB 00:00 2024-05-08T22:26:33.936 INFO:teuthology.orchestra.run.smithi066.stdout:(136/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 5.5 MB/s | 711 kB 00:00 2024-05-08T22:26:33.991 INFO:teuthology.orchestra.run.smithi062.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 3.4 MB/s | 854 kB 00:00 2024-05-08T22:26:34.066 INFO:teuthology.orchestra.run.smithi062.stdout:(135/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.9 MB/s | 711 kB 00:00 2024-05-08T22:26:34.158 INFO:teuthology.orchestra.run.smithi062.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.8 MB/s | 1.5 MB 00:00 2024-05-08T22:26:34.161 INFO:teuthology.orchestra.run.smithi066.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.3 MB/s | 333 kB 00:00 2024-05-08T22:26:34.191 INFO:teuthology.orchestra.run.smithi062.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.6 MB/s | 333 kB 00:00 2024-05-08T22:26:34.211 INFO:teuthology.orchestra.run.smithi066.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 3.9 MB/s | 1.1 MB 00:00 2024-05-08T22:26:34.250 INFO:teuthology.orchestra.run.smithi062.stdout:(138/146): librados2-18.2.2-1322.gd21e4f96.el8. 4.8 MB/s | 3.7 MB 00:00 2024-05-08T22:26:34.253 INFO:teuthology.orchestra.run.smithi066.stdout:(139/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 3.7 MB/s | 1.5 MB 00:00 2024-05-08T22:26:34.300 INFO:teuthology.orchestra.run.smithi062.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.5 MB/s | 1.1 MB 00:00 2024-05-08T22:26:34.320 INFO:teuthology.orchestra.run.smithi066.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.4 MB/s | 221 kB 00:00 2024-05-08T22:26:34.325 INFO:teuthology.orchestra.run.smithi062.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.6 MB/s | 221 kB 00:00 2024-05-08T22:26:34.354 INFO:teuthology.orchestra.run.smithi066.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 2.3 MB/s | 237 kB 00:00 2024-05-08T22:26:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.3 MB/s | 254 kB 00:00 2024-05-08T22:26:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:(142/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 3.0 MB/s | 180 kB 00:00 2024-05-08T22:26:34.387 INFO:teuthology.orchestra.run.smithi066.stdout:(142/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.4 MB/s | 254 kB 00:00 2024-05-08T22:26:34.463 INFO:teuthology.orchestra.run.smithi066.stdout:(143/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 914 kB/s | 99 kB 00:00 2024-05-08T22:26:34.476 INFO:teuthology.orchestra.run.smithi062.stdout:(143/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.3 MB/s | 237 kB 00:00 2024-05-08T22:26:34.496 INFO:teuthology.orchestra.run.smithi066.stdout:(144/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.0 MB/s | 180 kB 00:00 2024-05-08T22:26:34.552 INFO:teuthology.orchestra.run.smithi062.stdout:(144/146): util-linux-2.32.1-44.el8_9.1.x86_64. 15 MB/s | 2.5 MB 00:00 2024-05-08T22:26:34.571 INFO:teuthology.orchestra.run.smithi066.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 14 MB/s | 2.5 MB 00:00 2024-05-08T22:26:34.605 INFO:teuthology.orchestra.run.smithi066.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 725 kB/s | 103 kB 00:00 2024-05-08T22:26:34.609 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:26:34.609 INFO:teuthology.orchestra.run.smithi066.stdout:Total 12 MB/s | 215 MB 00:18 2024-05-08T22:26:34.660 INFO:teuthology.orchestra.run.smithi062.stdout:(145/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 330 kB/s | 99 kB 00:00 2024-05-08T22:26:34.685 INFO:teuthology.orchestra.run.smithi062.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 493 kB/s | 103 kB 00:00 2024-05-08T22:26:34.691 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:26:34.691 INFO:teuthology.orchestra.run.smithi062.stdout:Total 12 MB/s | 215 MB 00:18 2024-05-08T22:26:35.678 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-05-08T22:26:35.783 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-05-08T22:26:36.105 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-05-08T22:26:36.105 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-05-08T22:26:36.210 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-05-08T22:26:36.210 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-05-08T22:26:37.380 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-05-08T22:26:37.391 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-05-08T22:26:37.493 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-05-08T22:26:37.505 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-05-08T22:26:38.737 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-05-08T22:26:38.871 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T22:26:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:26:39.322 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:26:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T22:26:39.707 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:26:39.786 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:26:39.949 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:26:40.022 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:26:40.212 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T22:26:40.415 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-08T22:26:40.632 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T22:26:40.737 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T22:26:40.915 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:26:41.313 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:26:41.515 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:26:41.891 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:26:42.049 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:26:42.279 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:26:42.480 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:26:42.575 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:26:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:26:42.920 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-05-08T22:26:43.062 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T22:26:43.173 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:26:43.249 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:26:43.298 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-05-08T22:26:43.439 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T22:26:43.441 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:26:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T22:26:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:26:43.704 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:26:43.710 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T22:26:43.808 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T22:26:43.918 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T22:26:43.926 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T22:26:43.978 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:26:44.088 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T22:26:44.091 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:26:44.258 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:26:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:26:44.286 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T22:26:44.335 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:26:44.346 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T22:26:44.518 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T22:26:44.528 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T22:26:44.550 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:26:44.690 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:26:44.695 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T22:26:44.732 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T22:26:44.743 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T22:26:44.794 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:26:44.924 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T22:26:44.960 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T22:26:44.981 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:26:45.057 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:26:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:26:45.133 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T22:26:45.199 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T22:26:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-08T22:26:45.439 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T22:26:45.540 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-08T22:26:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:26:45.625 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:26:45.712 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:26:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-08T22:26:45.939 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T22:26:46.039 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T22:26:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T22:26:46.123 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:26:46.206 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T22:26:46.250 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:26:46.287 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:26:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:26:46.628 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:26:46.665 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:26:46.709 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:26:46.833 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:26:46.850 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T22:26:46.960 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:26:47.014 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T22:26:47.048 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:26:47.165 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:26:47.262 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:26:47.303 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:26:47.383 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:26:47.578 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T22:26:47.607 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T22:26:47.609 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:26:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T22:26:47.866 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:26:47.953 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T22:26:48.108 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T22:26:48.158 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T22:26:48.300 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T22:26:48.417 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T22:26:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T22:26:48.688 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T22:26:48.720 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-08T22:26:48.738 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T22:26:48.916 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T22:26:48.944 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-08T22:26:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:26:49.137 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T22:26:49.169 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:26:49.182 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T22:26:49.336 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:26:49.354 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T22:26:49.361 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T22:26:49.414 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:26:49.507 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T22:26:49.541 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:26:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T22:26:49.648 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:26:49.729 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T22:26:49.784 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T22:26:49.808 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:26:49.885 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T22:26:49.894 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T22:26:49.980 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T22:26:50.041 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T22:26:50.175 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T22:26:50.374 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T22:26:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T22:26:50.736 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T22:26:51.023 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T22:26:51.199 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T22:26:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T22:26:51.536 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T22:26:51.722 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T22:26:51.862 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T22:26:52.016 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T22:26:52.085 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-08T22:26:52.266 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T22:26:52.509 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T22:26:52.659 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T22:26:52.744 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T22:26:52.815 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T22:26:53.022 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T22:26:53.032 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-08T22:26:53.068 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T22:26:53.173 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T22:26:53.337 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T22:26:53.338 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T22:26:53.360 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-08T22:26:53.583 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T22:26:53.598 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T22:26:53.628 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-08T22:26:53.743 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T22:26:53.867 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T22:26:53.883 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T22:26:53.883 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T22:26:54.114 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T22:26:54.121 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T22:26:54.136 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T22:26:54.281 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T22:26:54.293 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T22:26:54.427 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T22:26:54.453 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T22:26:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:26:54.689 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T22:26:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:26:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:26:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:26:55.099 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T22:26:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T22:26:55.425 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T22:26:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T22:26:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:26:55.698 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T22:26:55.717 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T22:26:55.907 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T22:26:56.071 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T22:26:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:26:56.347 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T22:26:56.396 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:26:56.424 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:26:56.581 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T22:26:56.629 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T22:26:56.696 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T22:26:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-08T22:26:57.204 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-08T22:26:57.370 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T22:26:57.440 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T22:26:57.618 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-08T22:26:57.777 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-08T22:26:57.779 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T22:26:57.988 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-08T22:26:58.095 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T22:26:58.183 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/162 2024-05-08T22:26:58.316 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T22:26:58.378 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/162 2024-05-08T22:26:58.448 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T22:26:58.533 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/162 2024-05-08T22:26:58.557 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T22:26:58.715 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/162 2024-05-08T22:26:58.750 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T22:26:58.778 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T22:26:58.902 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T22:26:58.933 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/162 2024-05-08T22:26:59.038 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T22:26:59.094 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:26:59.173 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:26:59.234 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/162 2024-05-08T22:26:59.253 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T22:26:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:26:59.463 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T22:26:59.476 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/162 2024-05-08T22:26:59.508 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:26:59.640 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T22:26:59.725 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/162 2024-05-08T22:26:59.744 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T22:26:59.873 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:26:59.882 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/162 2024-05-08T22:26:59.939 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T22:26:59.955 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T22:27:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/162 2024-05-08T22:27:00.148 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:27:00.183 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:27:00.225 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T22:27:00.448 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T22:27:00.468 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/162 2024-05-08T22:27:00.638 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T22:27:00.639 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:27:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 67/162 2024-05-08T22:27:00.832 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:27:00.902 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:27:00.919 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:27:01.053 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 68/162 2024-05-08T22:27:01.131 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T22:27:01.290 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T22:27:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 69/162 2024-05-08T22:27:01.341 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T22:27:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 70/162 2024-05-08T22:27:01.524 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T22:27:01.582 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:27:01.600 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T22:27:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 71/162 2024-05-08T22:27:01.690 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T22:27:01.778 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T22:27:01.873 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T22:27:01.923 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 72/162 2024-05-08T22:27:01.972 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T22:27:02.162 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T22:27:02.170 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 73/162 2024-05-08T22:27:02.182 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T22:27:02.343 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T22:27:02.355 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/162 2024-05-08T22:27:02.430 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T22:27:02.540 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/162 2024-05-08T22:27:02.570 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T22:27:02.640 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T22:27:02.807 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T22:27:02.874 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T22:27:02.945 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/162 2024-05-08T22:27:03.065 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T22:27:03.136 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T22:27:03.144 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T22:27:03.301 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T22:27:03.412 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T22:27:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T22:27:03.480 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T22:27:03.572 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T22:27:03.714 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T22:27:03.735 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T22:27:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T22:27:03.967 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T22:27:04.018 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T22:27:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T22:27:04.161 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T22:27:04.218 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T22:27:04.287 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T22:27:04.387 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T22:27:04.460 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T22:27:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T22:27:04.626 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T22:27:04.704 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T22:27:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T22:27:04.882 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T22:27:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T22:27:04.956 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T22:27:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T22:27:05.130 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T22:27:05.146 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T22:27:05.270 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T22:27:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T22:27:05.358 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T22:27:05.478 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T22:27:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T22:27:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T22:27:05.739 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T22:27:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T22:27:05.825 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T22:27:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T22:27:05.926 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T22:27:06.006 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T22:27:06.100 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T22:27:06.171 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T22:27:06.271 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-05-08T22:27:06.320 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T22:27:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T22:27:06.494 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-05-08T22:27:06.561 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T22:27:06.599 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T22:27:06.676 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T22:27:06.748 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T22:27:06.749 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T22:27:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T22:27:06.834 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T22:27:06.925 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T22:27:07.109 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-05-08T22:27:07.122 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-05-08T22:27:07.298 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T22:27:07.324 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-05-08T22:27:07.514 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T22:27:07.520 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T22:27:07.587 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T22:27:07.660 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T22:27:07.798 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T22:27:07.825 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T22:27:07.905 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T22:27:07.989 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-05-08T22:27:08.104 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T22:27:08.186 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T22:27:08.213 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T22:27:08.367 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T22:27:08.400 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T22:27:08.425 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T22:27:08.539 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T22:27:08.623 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T22:27:08.701 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T22:27:08.746 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T22:27:08.828 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T22:27:08.945 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T22:27:09.018 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T22:27:09.133 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T22:27:09.190 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T22:27:09.209 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T22:27:09.346 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T22:27:09.403 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T22:27:09.577 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T22:27:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T22:27:09.749 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T22:27:09.751 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T22:27:09.951 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T22:27:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T22:27:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T22:27:10.142 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:27:10.162 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T22:27:10.188 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T22:27:10.212 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:27:10.365 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T22:27:10.410 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T22:27:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T22:27:10.601 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T22:27:10.628 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T22:27:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T22:27:10.867 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T22:27:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T22:27:11.093 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T22:27:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T22:27:11.345 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T22:27:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T22:27:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T22:27:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T22:27:12.089 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T22:27:12.350 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T22:27:12.494 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T22:27:12.841 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T22:27:13.026 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T22:27:13.310 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T22:27:13.490 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T22:27:13.576 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T22:27:13.786 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T22:27:13.810 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T22:27:14.023 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T22:27:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:27:14.308 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T22:27:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:27:14.556 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T22:27:14.638 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T22:27:14.770 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T22:27:14.825 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:27:14.895 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:27:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:27:15.075 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T22:27:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:27:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:27:15.400 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:27:15.583 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T22:27:15.746 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:27:15.824 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T22:27:19.024 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:27:19.837 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:27:20.025 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:27:20.054 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:27:20.660 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:27:20.683 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T22:27:20.954 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:27:20.966 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:27:21.023 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:27:21.052 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T22:27:21.158 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:27:21.195 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2024-05-08T22:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2024-05-08T22:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /dev 2024-05-08T22:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /run 2024-05-08T22:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2024-05-08T22:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2024-05-08T22:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2024-05-08T22:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2024-05-08T22:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /dev 2024-05-08T22:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:27:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:27:36.828 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:27:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:27:37.229 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:27:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:27:37.852 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:27:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:27:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:27:39.830 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:27:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:27:40.151 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T22:27:40.370 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T22:27:40.627 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:27:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:27:41.269 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2024-05-08T22:27:41.269 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2024-05-08T22:27:41.269 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /dev 2024-05-08T22:27:41.269 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /run 2024-05-08T22:27:41.269 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2024-05-08T22:27:41.269 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2024-05-08T22:27:41.269 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2024-05-08T22:27:41.269 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2024-05-08T22:27:41.269 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /dev 2024-05-08T22:27:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:27:41.569 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T22:27:41.570 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-05-08T22:27:41.570 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-05-08T22:27:41.570 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2024-05-08T22:27:41.570 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /run 2024-05-08T22:27:41.570 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-05-08T22:27:41.570 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-05-08T22:27:41.570 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-05-08T22:27:41.570 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-05-08T22:27:41.570 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2024-05-08T22:27:41.570 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:27:41.630 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:27:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:27:41.877 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:27:41.935 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:27:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:27:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:27:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:27:43.373 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:27:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T22:27:43.543 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:27:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:27:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:27:43.677 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T22:27:43.760 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:27:43.772 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:27:43.773 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:27:43.855 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:27:43.938 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:27:44.085 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T22:27:44.243 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:27:44.444 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:27:44.613 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:27:44.631 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:27:44.748 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T22:27:44.918 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:27:44.936 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T22:27:45.073 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T22:27:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:27:45.296 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:27:45.476 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:27:45.520 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:27:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:27:46.048 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:27:46.243 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:27:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:27:46.645 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:27:46.811 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T22:27:46.866 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:27:47.016 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T22:27:47.056 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T22:27:47.149 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T22:27:47.315 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:27:47.394 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T22:27:47.634 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:27:48.404 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:27:48.666 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:27:48.722 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T22:27:48.996 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:27:49.694 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:27:49.845 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:27:49.945 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:27:50.020 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T22:27:50.200 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:27:50.296 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T22:27:50.299 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T22:27:50.492 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:27:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:27:50.659 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T22:27:50.680 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:27:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:27:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:27:50.888 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:27:50.929 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T22:27:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:27:51.079 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T22:27:51.623 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T22:27:51.913 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-08T22:27:52.181 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T22:27:52.199 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-08T22:27:52.350 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:27:52.385 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:27:52.426 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T22:27:52.562 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:27:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 129/162 2024-05-08T22:27:52.595 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:27:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:27:52.631 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T22:27:52.762 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:27:52.825 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:27:52.852 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-08T22:27:52.869 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T22:27:53.058 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:27:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T22:27:53.130 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:27:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T22:27:53.343 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T22:27:53.387 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:27:53.428 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T22:27:53.545 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T22:27:53.625 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:27:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T22:27:55.969 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:27:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:27:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:27:56.728 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T22:27:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T22:27:57.361 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T22:27:57.515 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T22:27:57.732 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T22:27:58.010 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T22:27:58.128 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:27:58.318 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T22:27:58.364 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T22:27:58.540 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T22:27:58.617 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T22:27:58.737 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T22:27:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T22:27:58.867 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T22:27:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T22:27:59.023 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T22:27:59.032 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T22:27:59.033 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T22:27:59.163 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T22:27:59.175 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-08T22:27:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T22:27:59.387 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T22:27:59.401 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T22:27:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T22:27:59.728 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T22:27:59.728 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:27:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:27:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:28:00.134 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T22:28:00.149 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T22:28:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:28:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:28:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:28:00.437 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T22:28:00.457 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T22:28:00.658 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T22:28:00.748 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T22:28:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:28:00.854 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T22:28:01.047 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T22:28:01.050 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T22:28:01.050 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:28:01.050 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T22:28:01.245 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T22:28:01.264 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T22:28:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:28:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:28:01.429 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T22:28:01.440 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T22:28:01.542 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:28:01.542 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:28:01.625 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T22:28:01.628 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T22:28:01.792 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:28:01.792 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:28:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:28:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:28:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:28:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T22:28:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:28:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:28:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T22:28:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T22:28:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T22:28:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T22:28:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T22:28:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T22:28:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T22:28:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T22:28:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T22:28:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T22:28:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T22:28:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T22:28:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T22:28:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T22:28:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T22:28:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T22:28:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T22:28:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T22:28:03.331 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T22:28:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T22:28:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T22:28:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/162 2024-05-08T22:28:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 93/162 2024-05-08T22:28:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 94/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 95/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 96/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 97/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 98/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 99/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-08T22:28:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-08T22:28:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T22:28:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T22:28:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:28:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T22:28:04.036 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:28:04.060 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:28:04.324 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:28:04.344 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:28:04.361 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:28:04.385 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T22:28:04.784 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T22:28:04.825 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T22:28:05.099 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T22:28:05.169 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T22:28:05.328 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T22:28:05.390 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T22:28:05.507 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T22:28:05.597 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T22:28:05.691 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T22:28:05.794 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T22:28:05.950 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T22:28:06.070 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T22:28:06.216 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T22:28:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T22:28:06.389 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T22:28:06.400 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T22:28:06.502 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T22:28:06.513 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T22:28:06.615 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T22:28:06.701 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T22:28:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T22:28:06.901 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T22:28:06.933 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T22:28:06.987 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T22:28:07.106 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T22:28:07.120 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T22:28:07.202 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T22:28:07.216 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T22:28:07.282 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T22:28:07.436 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T22:28:07.458 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T22:28:07.619 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T22:28:07.619 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:28:07.629 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T22:28:07.798 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T22:28:07.798 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:28:07.893 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:28:07.893 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:28:08.063 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T22:28:08.064 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:28:08.143 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:28:08.143 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:28:08.314 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T22:28:08.314 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:28:08.372 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:28:08.542 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T22:28:08.608 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T22:28:08.737 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T22:28:08.760 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:28:08.907 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:28:08.983 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T22:28:08.983 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:28:09.137 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T22:28:09.137 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:28:09.184 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:28:09.184 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:28:09.371 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:28:09.371 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:28:09.451 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:28:09.451 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:28:09.596 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T22:28:09.596 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:28:09.684 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:28:09.684 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:28:09.804 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:28:09.805 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:28:09.950 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:28:09.950 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:28:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T22:28:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:28:10.175 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:28:10.271 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:28:10.367 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T22:28:10.367 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:28:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T22:28:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:28:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:28:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T22:28:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T22:28:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T22:28:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T22:28:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T22:28:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T22:28:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T22:28:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T22:28:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:28:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:28:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:28:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:28:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T22:28:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T22:28:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T22:28:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T22:28:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T22:28:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T22:28:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T22:28:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T22:28:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T22:28:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T22:28:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T22:28:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T22:28:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T22:28:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T22:28:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T22:28:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T22:28:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T22:28:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T22:28:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T22:28:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T22:28:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T22:28:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T22:28:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T22:28:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T22:28:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T22:28:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T22:28:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T22:28:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T22:28:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T22:28:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T22:28:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T22:28:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T22:28:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T22:28:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T22:28:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T22:28:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T22:28:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T22:28:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T22:28:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T22:28:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T22:28:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T22:28:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T22:28:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T22:28:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T22:28:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T22:28:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T22:28:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T22:28:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T22:28:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T22:28:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-05-08T22:28:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T22:28:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-05-08T22:28:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T22:28:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-05-08T22:28:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-05-08T22:28:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-05-08T22:28:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-05-08T22:28:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-05-08T22:28:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-05-08T22:28:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-05-08T22:28:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-05-08T22:28:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-05-08T22:28:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-05-08T22:28:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-05-08T22:28:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-05-08T22:28:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-05-08T22:28:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-05-08T22:28:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-05-08T22:28:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-05-08T22:28:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-05-08T22:28:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-05-08T22:28:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T22:28:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T22:28:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T22:28:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T22:28:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T22:28:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T22:28:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T22:28:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T22:28:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T22:28:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T22:28:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T22:28:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T22:28:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T22:28:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T22:28:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T22:28:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T22:28:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T22:28:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T22:28:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T22:28:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T22:28:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T22:28:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T22:28:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T22:28:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T22:28:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T22:28:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T22:28:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T22:28:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T22:28:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T22:28:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T22:28:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T22:28:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T22:28:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T22:28:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T22:28:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T22:28:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T22:28:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T22:28:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:28:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T22:28:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:28:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T22:28:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:28:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T22:28:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:28:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T22:28:11.367 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T22:28:11.367 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T22:28:11.367 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T22:28:11.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T22:28:11.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T22:28:11.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T22:28:11.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T22:28:11.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T22:28:11.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T22:28:11.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T22:28:11.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T22:28:11.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T22:28:11.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T22:28:11.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T22:28:11.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T22:28:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T22:28:11.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-05-08T22:28:11.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-05-08T22:28:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-05-08T22:28:11.375 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-05-08T22:28:11.375 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-05-08T22:28:11.375 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-05-08T22:28:11.375 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-05-08T22:28:11.375 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-05-08T22:28:11.375 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-05-08T22:28:11.375 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-05-08T22:28:11.375 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-05-08T22:28:11.375 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-05-08T22:28:11.375 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-05-08T22:28:11.375 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-05-08T22:28:11.377 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-05-08T22:28:11.377 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-05-08T22:28:11.377 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-05-08T22:28:11.377 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T22:28:11.377 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T22:28:11.377 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T22:28:11.377 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T22:28:11.377 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T22:28:11.377 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T22:28:11.378 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T22:28:11.379 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T22:28:11.379 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T22:28:11.379 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T22:28:11.379 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T22:28:11.379 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T22:28:11.379 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T22:28:11.379 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T22:28:11.379 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T22:28:11.379 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T22:28:11.379 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T22:28:11.380 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T22:28:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T22:28:11.382 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T22:28:11.382 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T22:28:11.382 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T22:28:16.362 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T22:28:16.362 INFO:teuthology.orchestra.run.smithi079.stdout:Installed products updated. 2024-05-08T22:28:16.977 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:28:16.977 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2024-05-08T22:28:16.977 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T22:28:16.977 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T22:28:16.977 INFO:teuthology.orchestra.run.smithi079.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T22:28:16.977 INFO:teuthology.orchestra.run.smithi079.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:16.978 INFO:teuthology.orchestra.run.smithi079.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:16.978 INFO:teuthology.orchestra.run.smithi079.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:16.978 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.978 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.978 INFO:teuthology.orchestra.run.smithi079.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:16.978 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.979 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T22:28:16.981 INFO:teuthology.orchestra.run.smithi079.stdout: jq-1.6-7.el8.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T22:28:16.982 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T22:28:16.983 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.983 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.983 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T22:28:16.983 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.983 INFO:teuthology.orchestra.run.smithi079.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T22:28:16.983 INFO:teuthology.orchestra.run.smithi079.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T22:28:16.983 INFO:teuthology.orchestra.run.smithi079.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T22:28:16.983 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T22:28:16.983 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:16.983 INFO:teuthology.orchestra.run.smithi079.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:28:16.984 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T22:28:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T22:28:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T22:28:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T22:28:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T22:28:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T22:28:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T22:28:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T22:28:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T22:28:16.986 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T22:28:16.986 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T22:28:16.986 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T22:28:16.986 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T22:28:16.986 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T22:28:16.986 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T22:28:16.986 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T22:28:16.986 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T22:28:16.986 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T22:28:16.986 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T22:28:16.987 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T22:28:16.987 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T22:28:16.987 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T22:28:16.987 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T22:28:16.987 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T22:28:16.987 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T22:28:16.987 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T22:28:16.987 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T22:28:16.987 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T22:28:16.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T22:28:16.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T22:28:16.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T22:28:16.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T22:28:16.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T22:28:16.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T22:28:16.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T22:28:16.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T22:28:16.989 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T22:28:16.990 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T22:28:16.991 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T22:28:16.991 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:28:16.991 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-05-08T22:28:19.688 DEBUG:teuthology.parallel:result is None 2024-05-08T22:28:21.028 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T22:28:21.028 INFO:teuthology.orchestra.run.smithi066.stdout:Installed products updated. 2024-05-08T22:28:21.683 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:28:21.684 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2024-05-08T22:28:21.684 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T22:28:21.684 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T22:28:21.684 INFO:teuthology.orchestra.run.smithi066.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T22:28:21.684 INFO:teuthology.orchestra.run.smithi066.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:21.684 INFO:teuthology.orchestra.run.smithi066.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:21.684 INFO:teuthology.orchestra.run.smithi066.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:21.684 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.684 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.684 INFO:teuthology.orchestra.run.smithi066.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:21.684 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.685 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:21.686 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T22:28:21.687 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: jq-1.6-7.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T22:28:21.688 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T22:28:21.689 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.689 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.689 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T22:28:21.689 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.689 INFO:teuthology.orchestra.run.smithi066.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T22:28:21.689 INFO:teuthology.orchestra.run.smithi066.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T22:28:21.689 INFO:teuthology.orchestra.run.smithi066.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T22:28:21.689 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T22:28:21.689 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:21.689 INFO:teuthology.orchestra.run.smithi066.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T22:28:21.689 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T22:28:21.690 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T22:28:21.691 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T22:28:21.691 INFO:teuthology.orchestra.run.smithi066.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T22:28:21.691 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T22:28:21.691 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.691 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.691 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.691 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T22:28:21.691 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T22:28:21.691 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T22:28:21.691 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T22:28:21.691 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T22:28:21.692 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.693 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T22:28:21.694 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T22:28:21.695 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T22:28:21.695 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T22:28:21.695 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T22:28:21.695 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.695 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.695 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:21.695 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T22:28:21.695 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T22:28:21.695 INFO:teuthology.orchestra.run.smithi066.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T22:28:21.695 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T22:28:21.695 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T22:28:21.696 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T22:28:21.696 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-08T22:28:21.696 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-05-08T22:28:21.900 DEBUG:teuthology.parallel:result is None 2024-05-08T22:28:24.032 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T22:28:24.032 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2024-05-08T22:28:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:28:24.689 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-05-08T22:28:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T22:28:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T22:28:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T22:28:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T22:28:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T22:28:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.6-7.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T22:28:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T22:28:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T22:28:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T22:28:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T22:28:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T22:28:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T22:28:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T22:28:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T22:28:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T22:28:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T22:28:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T22:28:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:28:24.700 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-05-08T22:28:24.898 DEBUG:teuthology.parallel:result is None 2024-05-08T22:28:24.899 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:28:25.055 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T22:28:25.081 INFO:teuthology.orchestra.run.smithi062.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T22:28:25.081 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T22:28:25.081 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T22:28:25.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:28:25.302 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T22:28:25.332 INFO:teuthology.orchestra.run.smithi066.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T22:28:25.332 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T22:28:25.332 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T22:28:25.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:28:25.479 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T22:28:25.509 INFO:teuthology.orchestra.run.smithi079.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T22:28:25.509 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T22:28:25.509 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T22:28:25.511 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T22:28:25.511 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:28:25.511 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T22:28:25.558 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-08T22:28:25.558 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T22:28:25.610 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-08T22:28:25.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T22:28:25.663 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T22:28:25.663 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:28:25.663 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T22:28:25.692 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T22:28:25.760 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-08T22:28:25.760 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T22:28:25.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T22:28:25.867 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-08T22:28:25.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T22:28:25.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T22:28:25.971 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T22:28:25.971 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:28:25.971 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T22:28:26.000 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T22:28:26.068 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-08T22:28:26.069 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T22:28:26.102 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T22:28:26.176 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-08T22:28:26.176 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T22:28:26.259 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T22:28:26.323 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T22:28:26.323 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:28:26.323 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T22:28:26.355 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T22:28:26.423 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-08T22:28:26.423 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T22:28:26.457 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T22:28:26.529 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-08T22:28:26.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T22:28:26.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T22:28:26.632 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T22:28:26.734 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'subvols': {'create': 2}} 2024-05-08T22:28:26.734 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:28:26.735 INFO:tasks.cephadm:Cluster fsid is 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:28:26.735 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T22:28:26.735 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.62', 'mon.b': '172.21.15.66', 'mon.c': '172.21.15.79'} 2024-05-08T22:28:26.735 INFO:tasks.cephadm:First mon is mon.a on smithi062 2024-05-08T22:28:26.735 INFO:tasks.cephadm:First mgr is x 2024-05-08T22:28:26.735 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T22:28:26.736 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2024-05-08T22:28:26.769 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname $(hostname -s) 2024-05-08T22:28:26.803 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname $(hostname -s) 2024-05-08T22:28:26.835 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T22:28:26.835 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:28:26.986 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79222/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:11:17.581213', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['arm64', 'source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79220/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 18:26:58.979379', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T22:28:27.130 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240508.150423-reef, sha1 d21e4f961318b56f0d915caa2490c3711bf13c34 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:28:27.131 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm 2024-05-08T22:28:27.131 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T22:28:27.652 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 22:28 /home/ubuntu/cephtest/cephadm 2024-05-08T22:28:27.652 DEBUG:teuthology.orchestra.run.smithi066:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T22:28:28.144 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 22:28 /home/ubuntu/cephtest/cephadm 2024-05-08T22:28:28.145 DEBUG:teuthology.orchestra.run.smithi079:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T22:28:28.489 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 22:28 /home/ubuntu/cephtest/cephadm 2024-05-08T22:28:28.489 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T22:28:28.512 DEBUG:teuthology.orchestra.run.smithi066:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T22:28:28.534 DEBUG:teuthology.orchestra.run.smithi079:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T22:28:28.572 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 on all hosts... 2024-05-08T22:28:28.572 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T22:28:28.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T22:28:28.577 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T22:28:29.614 INFO:teuthology.orchestra.run.smithi079.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T22:28:31.144 INFO:teuthology.orchestra.run.smithi066.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T22:28:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T22:28:58.193 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:28:58.193 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:28:58.193 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T22:28:58.193 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2024-05-08T22:28:58.193 INFO:teuthology.orchestra.run.smithi062.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T22:28:58.194 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-05-08T22:28:58.194 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:29:06.287 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-05-08T22:29:06.287 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:29:06.287 INFO:teuthology.orchestra.run.smithi066.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T22:29:06.287 INFO:teuthology.orchestra.run.smithi066.stdout: "repo_digests": [ 2024-05-08T22:29:06.287 INFO:teuthology.orchestra.run.smithi066.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T22:29:06.287 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-05-08T22:29:06.287 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-05-08T22:29:08.225 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-05-08T22:29:08.225 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T22:29:08.225 INFO:teuthology.orchestra.run.smithi079.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T22:29:08.225 INFO:teuthology.orchestra.run.smithi079.stdout: "repo_digests": [ 2024-05-08T22:29:08.225 INFO:teuthology.orchestra.run.smithi079.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T22:29:08.226 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-05-08T22:29:08.226 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-05-08T22:29:08.245 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2024-05-08T22:29:08.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph 2024-05-08T22:29:08.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph 2024-05-08T22:29:08.354 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2024-05-08T22:29:08.386 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /etc/ceph 2024-05-08T22:29:08.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /etc/ceph 2024-05-08T22:29:08.446 INFO:tasks.cephadm:Writing seed config... 2024-05-08T22:29:08.447 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T22:29:08.447 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T22:29:08.447 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T22:29:08.447 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T22:29:08.447 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T22:29:08.447 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-08T22:29:08.447 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T22:29:08.447 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T22:29:08.448 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T22:29:08.448 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T22:29:08.448 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T22:29:08.448 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T22:29:08.448 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T22:29:08.448 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T22:29:08.448 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T22:29:08.448 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T22:29:08.448 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T22:29:08.448 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T22:29:08.449 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T22:29:08.449 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T22:29:08.449 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T22:29:08.449 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T22:29:08.449 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T22:29:08.449 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T22:29:08.450 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T22:29:08.450 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T22:29:08.450 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T22:29:08.450 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-08T22:29:08.450 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-08T22:29:08.450 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-08T22:29:08.450 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T22:29:08.450 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T22:29:08.450 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-08T22:29:08.450 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-08T22:29:08.450 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T22:29:08.451 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T22:29:08.451 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-08T22:29:08.451 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T22:29:08.451 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T22:29:08.451 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T22:29:08.451 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T22:29:08.451 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T22:29:08.451 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T22:29:08.451 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T22:29:08.451 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-08T22:29:08.452 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:29:08.452 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T22:29:08.472 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 49d88ea8-0d8a-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T22:29:08.472 DEBUG:teuthology.orchestra.run.smithi062:mon.a> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T22:29:08.516 DEBUG:teuthology.orchestra.run.smithi062:mgr.x> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T22:29:08.546 INFO:journalctl@ceph.mon.a.smithi062.stdout:-- Logs begin at Wed 2024-05-08 22:12:07 UTC. -- 2024-05-08T22:29:08.547 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T22:29:08.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 -v bootstrap --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.62 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:29:08.575 INFO:journalctl@ceph.mgr.x.smithi062.stdout:-- Logs begin at Wed 2024-05-08 22:12:07 UTC. -- 2024-05-08T22:29:08.719 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-05-08T22:29:08.719 INFO:teuthology.orchestra.run.smithi062.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34', '-v', 'bootstrap', '--fsid', '49d88ea8-0d8a-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.62', '--skip-admin-label'] 2024-05-08T22:29:08.749 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 4.6.1 2024-05-08T22:29:08.749 INFO:teuthology.orchestra.run.smithi062.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T22:29:08.749 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying podman|docker is present... 2024-05-08T22:29:08.786 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 4.6.1 2024-05-08T22:29:08.786 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying lvm2 is present... 2024-05-08T22:29:08.786 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying time synchronization is in place... 2024-05-08T22:29:08.793 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T22:29:08.793 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T22:29:08.799 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T22:29:08.799 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-05-08T22:29:08.805 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2024-05-08T22:29:08.811 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2024-05-08T22:29:08.811 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2024-05-08T22:29:08.812 INFO:teuthology.orchestra.run.smithi062.stdout:Repeating the final host check... 2024-05-08T22:29:08.843 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 4.6.1 2024-05-08T22:29:08.843 INFO:teuthology.orchestra.run.smithi062.stdout:podman (/bin/podman) version 4.6.1 is present 2024-05-08T22:29:08.843 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl is present 2024-05-08T22:29:08.843 INFO:teuthology.orchestra.run.smithi062.stdout:lvcreate is present 2024-05-08T22:29:08.850 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T22:29:08.850 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T22:29:08.855 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T22:29:08.855 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-05-08T22:29:08.861 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2024-05-08T22:29:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2024-05-08T22:29:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2024-05-08T22:29:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:Host looks OK 2024-05-08T22:29:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:Cluster fsid: 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:29:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:Acquiring lock 140025419684328 on /run/cephadm/49d88ea8-0d8a-11ef-bc97-c7b262605968.lock 2024-05-08T22:29:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:Lock 140025419684328 acquired on /run/cephadm/49d88ea8-0d8a-11ef-bc97-c7b262605968.lock 2024-05-08T22:29:08.868 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 3300 ... 2024-05-08T22:29:08.868 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 6789 ... 2024-05-08T22:29:08.868 INFO:teuthology.orchestra.run.smithi062.stdout:Base mon IP(s) is [172.21.15.62:3300, 172.21.15.62:6789], mon addrv is [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-05-08T22:29:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2024-05-08T22:29:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.62 metric 100 2024-05-08T22:29:08.875 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T22:29:08.875 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-05-08T22:29:08.875 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-05-08T22:29:08.877 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T22:29:08.878 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T22:29:08.878 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T22:29:08.878 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-08T22:29:08.878 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 fe80::2bc6:2bcc:607:37e4/64 scope link noprefixroute 2024-05-08T22:29:08.878 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T22:29:08.878 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2024-05-08T22:29:08.879 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2024-05-08T22:29:08.879 INFO:teuthology.orchestra.run.smithi062.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T22:29:08.879 INFO:teuthology.orchestra.run.smithi062.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T22:29:08.880 INFO:teuthology.orchestra.run.smithi062.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T22:29:09.337 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T22:29:09.337 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T22:29:09.337 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T22:29:09.337 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T22:29:09.337 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:7850648fa4707607dddc5eec7674b8c9296086cf87f20a8b1acf1b014b952e9f 2024-05-08T22:29:09.337 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying config sha256:af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T22:29:09.337 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-08T22:29:10.031 INFO:teuthology.orchestra.run.smithi062.stdout:ceph: stdout ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T22:29:10.031 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph version: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T22:29:10.031 INFO:teuthology.orchestra.run.smithi062.stdout:Extracting ceph user uid/gid from container image... 2024-05-08T22:29:10.645 INFO:teuthology.orchestra.run.smithi062.stdout:stat: stdout 167 167 2024-05-08T22:29:10.645 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial keys... 2024-05-08T22:29:11.290 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQC2/DtmRTT8NRAAACOnsUkRi9y6yruo3CRxtQ== 2024-05-08T22:29:11.898 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQC3/DtmK/OhIBAAxPb8lHMYcHTNEv5z6vwEjA== 2024-05-08T22:29:12.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQC4/Dtmz25ODRAA7xwwUgqRiJ0GrvtFr5pdhw== 2024-05-08T22:29:12.577 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial monmap... 2024-05-08T22:29:14.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T22:29:14.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-08T22:29:14.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:29:14.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T22:29:14.368 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool for a [v2:172.21.15.62:3300,v1:172.21.15.62:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T22:29:14.368 INFO:teuthology.orchestra.run.smithi062.stdout:setting min_mon_release = pacific 2024-05-08T22:29:14.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: set fsid to 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:29:14.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T22:29:14.368 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:29:14.368 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mon... 2024-05-08T22:29:15.756 INFO:teuthology.orchestra.run.smithi062.stdout:create mon.a on 2024-05-08T22:29:15.899 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-08T22:29:15.998 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-08T22:29:16.123 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968.target. 2024-05-08T22:29:16.123 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968.target. 2024-05-08T22:29:16.307 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.a 2024-05-08T22:29:16.307 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-08T22:29:16.404 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968.target.wants/ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@.service. 2024-05-08T22:29:16.782 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T22:29:16.783 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-05-08T22:29:16.790 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T22:29:16.790 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-05-08T22:29:16.790 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-05-08T22:29:16.790 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T22:29:16.790 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon to start... 2024-05-08T22:29:16.791 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon... 2024-05-08T22:29:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:16 smithi062 ceph-mon[42092]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T22:29:17.565 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout cluster: 2024-05-08T22:29:17.565 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout id: 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout services: 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.447838s) 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout data: 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pgs: 2024-05-08T22:29:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.173+0000 7ff5b4a11700 1 Processor -- start 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.173+0000 7ff5b4a11700 1 -- start start 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.173+0000 7ff5b4a11700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0109670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.173+0000 7ff5b4a11700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0074760 con 0x7ff5b0107280 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.174+0000 7ff5ae59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0109670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.174+0000 7ff5ae59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0109670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44198/0 (socket says 172.21.15.62:44198) 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.174+0000 7ff5ae59c700 1 -- 172.21.15.62:0/2243650549 learned_addr learned my addr 172.21.15.62:0/2243650549 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.174+0000 7ff5ae59c700 1 -- 172.21.15.62:0/2243650549 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b00748a0 con 0x7ff5b0107280 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.174+0000 7ff5ae59c700 1 --2- 172.21.15.62:0/2243650549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0109670 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00098a0 tx=0x7ff5a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb8b588c84a5529 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.175+0000 7ff5ad59a700 1 -- 172.21.15.62:0/2243650549 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5a0004160 con 0x7ff5b0107280 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.175+0000 7ff5ad59a700 1 -- 172.21.15.62:0/2243650549 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a00042c0 con 0x7ff5b0107280 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.175+0000 7ff5ad59a700 1 -- 172.21.15.62:0/2243650549 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5a00045a0 con 0x7ff5b0107280 2024-05-08T22:29:17.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.175+0000 7ff5b4a11700 1 -- 172.21.15.62:0/2243650549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 msgr2=0x7ff5b0109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.175+0000 7ff5b4a11700 1 --2- 172.21.15.62:0/2243650549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0109670 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00098a0 tx=0x7ff5a0009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.175+0000 7ff5b4a11700 1 -- 172.21.15.62:0/2243650549 shutdown_connections 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.175+0000 7ff5b4a11700 1 --2- 172.21.15.62:0/2243650549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0109670 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.175+0000 7ff5b4a11700 1 -- 172.21.15.62:0/2243650549 >> 172.21.15.62:0/2243650549 conn(0x7ff5b0100bd0 msgr2=0x7ff5b0103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.175+0000 7ff5b4a11700 1 -- 172.21.15.62:0/2243650549 shutdown_connections 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.175+0000 7ff5b4a11700 1 -- 172.21.15.62:0/2243650549 wait complete. 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.176+0000 7ff5b4a11700 1 Processor -- start 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.176+0000 7ff5b4a11700 1 -- start start 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.176+0000 7ff5b4a11700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0106a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.176+0000 7ff5b4a11700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0106fa0 con 0x7ff5b0107280 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.176+0000 7ff5ae59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:17.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.176+0000 7ff5ae59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44206/0 (socket says 172.21.15.62:44206) 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.176+0000 7ff5ae59c700 1 -- 172.21.15.62:0/1366075249 learned_addr learned my addr 172.21.15.62:0/1366075249 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.177+0000 7ff5ae59c700 1 -- 172.21.15.62:0/1366075249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a0009550 con 0x7ff5b0107280 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.177+0000 7ff5ae59c700 1 --2- 172.21.15.62:0/1366075249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0106a60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0073ba0 tx=0x7ff5a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.177+0000 7ff59f7fe700 1 -- 172.21.15.62:0/1366075249 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5a0011630 con 0x7ff5b0107280 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.177+0000 7ff59f7fe700 1 -- 172.21.15.62:0/1366075249 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a0011790 con 0x7ff5b0107280 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.177+0000 7ff5b4a11700 1 -- 172.21.15.62:0/1366075249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b0105230 con 0x7ff5b0107280 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.177+0000 7ff59f7fe700 1 -- 172.21.15.62:0/1366075249 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5a0011a30 con 0x7ff5b0107280 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.177+0000 7ff5b4a11700 1 -- 172.21.15.62:0/1366075249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b01056d0 con 0x7ff5b0107280 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.178+0000 7ff59f7fe700 1 -- 172.21.15.62:0/1366075249 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff5a0021070 con 0x7ff5b0107280 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.178+0000 7ff59f7fe700 1 -- 172.21.15.62:0/1366075249 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff5a001b780 con 0x7ff5b0107280 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.178+0000 7ff5b4a11700 1 -- 172.21.15.62:0/1366075249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b01060e0 con 0x7ff5b0107280 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.180+0000 7ff59f7fe700 1 -- 172.21.15.62:0/1366075249 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff5a001acc0 con 0x7ff5b0107280 2024-05-08T22:29:17.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.254+0000 7ff5b4a11700 1 -- 172.21.15.62:0/1366075249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff5b0109d90 con 0x7ff5b0107280 2024-05-08T22:29:17.570 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.254+0000 7ff59f7fe700 1 -- 172.21.15.62:0/1366075249 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7ff5a001ae60 con 0x7ff5b0107280 2024-05-08T22:29:17.570 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.255+0000 7ff5b4a11700 1 -- 172.21.15.62:0/1366075249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 msgr2=0x7ff5b0106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:17.570 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.255+0000 7ff5b4a11700 1 --2- 172.21.15.62:0/1366075249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0106a60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0073ba0 tx=0x7ff5a0004750 comp rx=0 tx=0).stop 2024-05-08T22:29:17.570 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.255+0000 7ff5b4a11700 1 -- 172.21.15.62:0/1366075249 shutdown_connections 2024-05-08T22:29:17.570 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.255+0000 7ff5b4a11700 1 --2- 172.21.15.62:0/1366075249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5b0107280 0x7ff5b0106a60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:17.570 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.255+0000 7ff5b4a11700 1 -- 172.21.15.62:0/1366075249 >> 172.21.15.62:0/1366075249 conn(0x7ff5b0100bd0 msgr2=0x7ff5b0102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:17.570 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.255+0000 7ff5b4a11700 1 -- 172.21.15.62:0/1366075249 shutdown_connections 2024-05-08T22:29:17.570 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.255+0000 7ff5b4a11700 1 -- 172.21.15.62:0/1366075249 wait complete. 2024-05-08T22:29:17.570 INFO:teuthology.orchestra.run.smithi062.stdout:mon is available 2024-05-08T22:29:17.570 INFO:teuthology.orchestra.run.smithi062.stdout:Assimilating anything we can from ceph.conf... 2024-05-08T22:29:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:17 smithi062 ceph-mon[42092]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T22:29:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:17 smithi062 ceph-mon[42092]: monmap e1: 1 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:29:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:17 smithi062 ceph-mon[42092]: fsmap 2024-05-08T22:29:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:17 smithi062 ceph-mon[42092]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T22:29:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:17 smithi062 ceph-mon[42092]: mgrmap e1: no daemons active 2024-05-08T22:29:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:17 smithi062 ceph-mon[42092]: from='client.? 172.21.15.62:0/1366075249' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T22:29:18.386 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:18.386 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T22:29:18.386 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T22:29:18.387 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T22:29:18.388 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T22:29:18.388 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.904+0000 7fd43bc33700 1 Processor -- start 2024-05-08T22:29:18.388 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.905+0000 7fd43bc33700 1 -- start start 2024-05-08T22:29:18.388 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.905+0000 7fd43bc33700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd434105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:18.388 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.905+0000 7fd43bc33700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd434079e30 con 0x7fd434105010 2024-05-08T22:29:18.388 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.905+0000 7fd4399cf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd434105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:18.388 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.905+0000 7fd4399cf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd434105430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44218/0 (socket says 172.21.15.62:44218) 2024-05-08T22:29:18.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.905+0000 7fd4399cf700 1 -- 172.21.15.62:0/2450771751 learned_addr learned my addr 172.21.15.62:0/2450771751 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:18.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.905+0000 7fd4399cf700 1 -- 172.21.15.62:0/2450771751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd434105970 con 0x7fd434105010 2024-05-08T22:29:18.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.906+0000 7fd4399cf700 1 --2- 172.21.15.62:0/2450771751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd434105430 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd4300098a0 tx=0x7fd430009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15785790c9d49ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:18.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.906+0000 7fd4389cd700 1 -- 172.21.15.62:0/2450771751 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd430004160 con 0x7fd434105010 2024-05-08T22:29:18.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.906+0000 7fd4389cd700 1 -- 172.21.15.62:0/2450771751 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4300042c0 con 0x7fd434105010 2024-05-08T22:29:18.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.906+0000 7fd4389cd700 1 -- 172.21.15.62:0/2450771751 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4300045a0 con 0x7fd434105010 2024-05-08T22:29:18.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.906+0000 7fd43bc33700 1 -- 172.21.15.62:0/2450771751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 msgr2=0x7fd434105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:18.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.906+0000 7fd43bc33700 1 --2- 172.21.15.62:0/2450771751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd434105430 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd4300098a0 tx=0x7fd430009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:18.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.906+0000 7fd43bc33700 1 -- 172.21.15.62:0/2450771751 shutdown_connections 2024-05-08T22:29:18.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.906+0000 7fd43bc33700 1 --2- 172.21.15.62:0/2450771751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd434105430 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:18.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.906+0000 7fd43bc33700 1 -- 172.21.15.62:0/2450771751 >> 172.21.15.62:0/2450771751 conn(0x7fd434100bb0 msgr2=0x7fd434103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:18.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.906+0000 7fd43bc33700 1 -- 172.21.15.62:0/2450771751 shutdown_connections 2024-05-08T22:29:18.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.906+0000 7fd43bc33700 1 -- 172.21.15.62:0/2450771751 wait complete. 2024-05-08T22:29:18.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.907+0000 7fd43bc33700 1 Processor -- start 2024-05-08T22:29:18.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.907+0000 7fd43bc33700 1 -- start start 2024-05-08T22:29:18.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.907+0000 7fd43bc33700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd43419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:18.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.907+0000 7fd43bc33700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43419c330 con 0x7fd434105010 2024-05-08T22:29:18.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.908+0000 7fd4399cf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd43419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:18.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.908+0000 7fd4399cf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd43419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44230/0 (socket says 172.21.15.62:44230) 2024-05-08T22:29:18.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.908+0000 7fd4399cf700 1 -- 172.21.15.62:0/402087504 learned_addr learned my addr 172.21.15.62:0/402087504 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:18.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.908+0000 7fd4399cf700 1 -- 172.21.15.62:0/402087504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd430009550 con 0x7fd434105010 2024-05-08T22:29:18.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.908+0000 7fd4399cf700 1 --2- 172.21.15.62:0/402087504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd43419bdf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd434079270 tx=0x7fd430004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:18.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.908+0000 7fd42affd700 1 -- 172.21.15.62:0/402087504 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd430011630 con 0x7fd434105010 2024-05-08T22:29:18.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.908+0000 7fd42affd700 1 -- 172.21.15.62:0/402087504 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd430011790 con 0x7fd434105010 2024-05-08T22:29:18.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.908+0000 7fd43bc33700 1 -- 172.21.15.62:0/402087504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43419c530 con 0x7fd434105010 2024-05-08T22:29:18.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.908+0000 7fd42affd700 1 -- 172.21.15.62:0/402087504 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd430011a30 con 0x7fd434105010 2024-05-08T22:29:18.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.908+0000 7fd43bc33700 1 -- 172.21.15.62:0/402087504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43419c9d0 con 0x7fd434105010 2024-05-08T22:29:18.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.909+0000 7fd42affd700 1 -- 172.21.15.62:0/402087504 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd430022070 con 0x7fd434105010 2024-05-08T22:29:18.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.909+0000 7fd42affd700 1 -- 172.21.15.62:0/402087504 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd43001c450 con 0x7fd434105010 2024-05-08T22:29:18.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.909+0000 7fd43bc33700 1 -- 172.21.15.62:0/402087504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd434105ce0 con 0x7fd434105010 2024-05-08T22:29:18.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.911+0000 7fd42affd700 1 -- 172.21.15.62:0/402087504 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fd43001b450 con 0x7fd434105010 2024-05-08T22:29:18.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.981+0000 7fd43bc33700 1 -- 172.21.15.62:0/402087504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd434105e90 con 0x7fd434105010 2024-05-08T22:29:18.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.985+0000 7fd42affd700 1 -- 172.21.15.62:0/402087504 <== mon.0 v2:172.21.15.62:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd43001b5f0 con 0x7fd434105010 2024-05-08T22:29:18.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.985+0000 7fd42affd700 1 -- 172.21.15.62:0/402087504 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fd430024630 con 0x7fd434105010 2024-05-08T22:29:18.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.986+0000 7fd43bc33700 1 -- 172.21.15.62:0/402087504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 msgr2=0x7fd43419bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:18.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.986+0000 7fd43bc33700 1 --2- 172.21.15.62:0/402087504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd43419bdf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd434079270 tx=0x7fd430004750 comp rx=0 tx=0).stop 2024-05-08T22:29:18.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.986+0000 7fd43bc33700 1 -- 172.21.15.62:0/402087504 shutdown_connections 2024-05-08T22:29:18.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.986+0000 7fd43bc33700 1 --2- 172.21.15.62:0/402087504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd434105010 0x7fd43419bdf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:18.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.986+0000 7fd43bc33700 1 -- 172.21.15.62:0/402087504 >> 172.21.15.62:0/402087504 conn(0x7fd434100bb0 msgr2=0x7fd4341925d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:18.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.986+0000 7fd43bc33700 1 -- 172.21.15.62:0/402087504 shutdown_connections 2024-05-08T22:29:18.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:17.986+0000 7fd43bc33700 1 -- 172.21.15.62:0/402087504 wait complete. 2024-05-08T22:29:18.393 INFO:teuthology.orchestra.run.smithi062.stdout:Generating new minimal ceph.conf... 2024-05-08T22:29:19.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:18 smithi062 ceph-mon[42092]: from='client.? 172.21.15.62:0/402087504' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T22:29:19.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:18 smithi062 ceph-mon[42092]: from='client.? 172.21.15.62:0/402087504' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T22:29:19.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:18 smithi062 ceph-mon[42092]: from='client.? 172.21.15.62:0/2608462915' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:29:19.166 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.720+0000 7f67af6b0700 1 Processor -- start 2024-05-08T22:29:19.166 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.720+0000 7f67af6b0700 1 -- start start 2024-05-08T22:29:19.166 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.720+0000 7f67af6b0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a81095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:19.166 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.720+0000 7f67af6b0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80747e0 con 0x7f67a8107200 2024-05-08T22:29:19.166 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.720+0000 7f67ad44c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a81095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:19.166 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.720+0000 7f67ad44c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a81095f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44242/0 (socket says 172.21.15.62:44242) 2024-05-08T22:29:19.166 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.720+0000 7f67ad44c700 1 -- 172.21.15.62:0/3617246749 learned_addr learned my addr 172.21.15.62:0/3617246749 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.721+0000 7f67ad44c700 1 -- 172.21.15.62:0/3617246749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8109b30 con 0x7f67a8107200 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.721+0000 7f67ad44c700 1 --2- 172.21.15.62:0/3617246749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a81095f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67a40098a0 tx=0x7f67a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b373527ae28d8dd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.721+0000 7f679ffff700 1 -- 172.21.15.62:0/3617246749 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67a4004160 con 0x7f67a8107200 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.721+0000 7f679ffff700 1 -- 172.21.15.62:0/3617246749 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f67a40042c0 con 0x7f67a8107200 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.721+0000 7f679ffff700 1 -- 172.21.15.62:0/3617246749 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67a4011a00 con 0x7f67a8107200 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.721+0000 7f67af6b0700 1 -- 172.21.15.62:0/3617246749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 msgr2=0x7f67a81095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.721+0000 7f67af6b0700 1 --2- 172.21.15.62:0/3617246749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a81095f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67a40098a0 tx=0x7f67a4009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.721+0000 7f67af6b0700 1 -- 172.21.15.62:0/3617246749 shutdown_connections 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.721+0000 7f67af6b0700 1 --2- 172.21.15.62:0/3617246749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a81095f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.721+0000 7f67af6b0700 1 -- 172.21.15.62:0/3617246749 >> 172.21.15.62:0/3617246749 conn(0x7f67a8100bd0 msgr2=0x7f67a8103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.722+0000 7f67af6b0700 1 -- 172.21.15.62:0/3617246749 shutdown_connections 2024-05-08T22:29:19.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.722+0000 7f67af6b0700 1 -- 172.21.15.62:0/3617246749 wait complete. 2024-05-08T22:29:19.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.722+0000 7f67af6b0700 1 Processor -- start 2024-05-08T22:29:19.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.722+0000 7f67af6b0700 1 -- start start 2024-05-08T22:29:19.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.722+0000 7f67af6b0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:19.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.722+0000 7f67af6b0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a819aa10 con 0x7f67a8107200 2024-05-08T22:29:19.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.722+0000 7f67ad44c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:19.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.722+0000 7f67ad44c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44250/0 (socket says 172.21.15.62:44250) 2024-05-08T22:29:19.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.723+0000 7f67ad44c700 1 -- 172.21.15.62:0/2608462915 learned_addr learned my addr 172.21.15.62:0/2608462915 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:19.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.723+0000 7f67ad44c700 1 -- 172.21.15.62:0/2608462915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a4009550 con 0x7f67a8107200 2024-05-08T22:29:19.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.723+0000 7f67ad44c700 1 --2- 172.21.15.62:0/2608462915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a819a4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67a4011a20 tx=0x7f67a4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:19.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.723+0000 7f679e7fc700 1 -- 172.21.15.62:0/2608462915 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67a4003af0 con 0x7f67a8107200 2024-05-08T22:29:19.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.723+0000 7f679e7fc700 1 -- 172.21.15.62:0/2608462915 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f67a4003c50 con 0x7f67a8107200 2024-05-08T22:29:19.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.723+0000 7f67af6b0700 1 -- 172.21.15.62:0/2608462915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a819ac10 con 0x7f67a8107200 2024-05-08T22:29:19.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.723+0000 7f679e7fc700 1 -- 172.21.15.62:0/2608462915 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67a4019760 con 0x7f67a8107200 2024-05-08T22:29:19.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.723+0000 7f67af6b0700 1 -- 172.21.15.62:0/2608462915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a819b0b0 con 0x7f67a8107200 2024-05-08T22:29:19.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.723+0000 7f679e7fc700 1 -- 172.21.15.62:0/2608462915 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f67a4019950 con 0x7f67a8107200 2024-05-08T22:29:19.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.723+0000 7f679e7fc700 1 -- 172.21.15.62:0/2608462915 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f67a400b3e0 con 0x7f67a8107200 2024-05-08T22:29:19.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.724+0000 7f67af6b0700 1 -- 172.21.15.62:0/2608462915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a810a8d0 con 0x7f67a8107200 2024-05-08T22:29:19.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.726+0000 7f679e7fc700 1 -- 172.21.15.62:0/2608462915 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f67a402a060 con 0x7f67a8107200 2024-05-08T22:29:19.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.796+0000 7f67af6b0700 1 -- 172.21.15.62:0/2608462915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f67a810aac0 con 0x7f67a8107200 2024-05-08T22:29:19.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.797+0000 7f679e7fc700 1 -- 172.21.15.62:0/2608462915 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f67a4019d90 con 0x7f67a8107200 2024-05-08T22:29:19.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.797+0000 7f67af6b0700 1 -- 172.21.15.62:0/2608462915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 msgr2=0x7f67a819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:19.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.797+0000 7f67af6b0700 1 --2- 172.21.15.62:0/2608462915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a819a4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67a4011a20 tx=0x7f67a4011b00 comp rx=0 tx=0).stop 2024-05-08T22:29:19.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.797+0000 7f67af6b0700 1 -- 172.21.15.62:0/2608462915 shutdown_connections 2024-05-08T22:29:19.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.797+0000 7f67af6b0700 1 --2- 172.21.15.62:0/2608462915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a8107200 0x7f67a819a4d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:19.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.797+0000 7f67af6b0700 1 -- 172.21.15.62:0/2608462915 >> 172.21.15.62:0/2608462915 conn(0x7f67a8100bd0 msgr2=0x7f67a81017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:19.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.797+0000 7f67af6b0700 1 -- 172.21.15.62:0/2608462915 shutdown_connections 2024-05-08T22:29:19.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:18.798+0000 7f67af6b0700 1 -- 172.21.15.62:0/2608462915 wait complete. 2024-05-08T22:29:19.170 INFO:teuthology.orchestra.run.smithi062.stdout:Restarting the monitor... 2024-05-08T22:29:19.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:19 smithi062 systemd[1]: Stopping Ceph mon.a for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:29:19.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:19 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-a[42069]: 2024-05-08T22:29:19.373+0000 7f13302fc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:29:20.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:19 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-a[42069]: 2024-05-08T22:29:19.373+0000 7f13302fc700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-08T22:29:20.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:19 smithi062 bash[42416]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-a 2024-05-08T22:29:20.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:19 smithi062 systemd[1]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.a.service: Succeeded. 2024-05-08T22:29:20.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:19 smithi062 systemd[1]: Stopped Ceph mon.a for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:29:20.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:19 smithi062 systemd[1]: Starting Ceph mon.a for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:29:20.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T22:29:20.236 INFO:teuthology.orchestra.run.smithi062.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-08T22:29:20.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable), process ceph-mon, pid 7 2024-05-08T22:29:20.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: pidfile_write: ignore empty --pid-file 2024-05-08T22:29:20.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 bash[42534]: 2b83cc9886cd34f395629cba38232f9563970b41d584113e56575e8ab5d88b13 2024-05-08T22:29:20.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 systemd[1]: Started Ceph mon.a for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:29:20.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: load: jerasure load: lrc 2024-05-08T22:29:20.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: RocksDB version: 7.9.2 2024-05-08T22:29:20.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Git sha 0 2024-05-08T22:29:20.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Compile date 2024-05-08 17:55:50 2024-05-08T22:29:20.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: DB SUMMARY 2024-05-08T22:29:20.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: DB Session ID: WR5ND50L0C8J4HQ37RDG 2024-05-08T22:29:20.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: CURRENT file: CURRENT 2024-05-08T22:29:20.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T22:29:20.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-08T22:29:20.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-08T22:29:20.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98891 ; 2024-05-08T22:29:20.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.error_if_exists: 0 2024-05-08T22:29:20.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.create_if_missing: 0 2024-05-08T22:29:20.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.paranoid_checks: 1 2024-05-08T22:29:20.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T22:29:20.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T22:29:20.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T22:29:20.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.env: 0x562c49da7820 2024-05-08T22:29:20.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T22:29:20.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.info_log: 0x562c4c29d520 2024-05-08T22:29:20.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T22:29:20.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.statistics: (nil) 2024-05-08T22:29:20.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.use_fsync: 0 2024-05-08T22:29:20.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_log_file_size: 0 2024-05-08T22:29:20.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T22:29:20.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T22:29:20.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T22:29:20.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T22:29:20.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.allow_fallocate: 1 2024-05-08T22:29:20.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T22:29:20.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T22:29:20.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.use_direct_reads: 0 2024-05-08T22:29:20.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T22:29:20.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T22:29:20.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.db_log_dir: 2024-05-08T22:29:20.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.wal_dir: 2024-05-08T22:29:20.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T22:29:20.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T22:29:20.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T22:29:20.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T22:29:20.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.write_buffer_manager: 0x562c4b532320 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.unordered_write: 0 2024-05-08T22:29:20.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.row_cache: None 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.wal_filter: None 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.two_write_queues: 0 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.wal_compression: 0 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.atomic_flush: 0 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T22:29:20.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.log_readahead_size: 0 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_background_jobs: 2 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_background_compactions: -1 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_subcompactions: 1 2024-05-08T22:29:20.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_open_files: -1 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_background_flushes: -1 2024-05-08T22:29:20.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Compression algorithms supported: 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: kZSTD supported: 0 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: kXpressCompression supported: 0 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: kZlibCompression supported: 1 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: kSnappyCompression supported: 1 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: kLZ4Compression supported: 1 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: kBZip2Compression supported: 0 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-08T22:29:20.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.merge_operator: 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_filter: None 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562c4c29d620) 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_type: 0 2024-05-08T22:29:20.434 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_index_type: 0 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_shortening: 1 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: checksum: 4 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: no_block_cache: 0 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache: 0x562c4b5b51f0 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_options: 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: capacity : 536870912 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_shard_bits : 4 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: strict_capacity_limit : 0 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-05-08T22:29:20.435 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_compressed: (nil) 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: persistent_cache: (nil) 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size: 4096 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size_deviation: 10 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_restart_interval: 16 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_block_restart_interval: 1 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: metadata_block_size: 4096 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: partition_filters: 0 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: use_delta_encoding: 1 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: filter_policy: bloomfilter 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: whole_key_filtering: 1 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: verify_compression: 0 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: format_version: 5 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_index_compression: 1 2024-05-08T22:29:20.436 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_align: 0 2024-05-08T22:29:20.437 INFO:journalctl@ceph.mon.a.smithi062.stdout: max_auto_readahead_size: 262144 2024-05-08T22:29:20.437 INFO:journalctl@ceph.mon.a.smithi062.stdout: prepopulate_block_cache: 0 2024-05-08T22:29:20.437 INFO:journalctl@ceph.mon.a.smithi062.stdout: initial_auto_readahead_size: 8192 2024-05-08T22:29:20.437 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T22:29:20.437 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T22:29:20.437 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T22:29:20.437 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compression: NoCompression 2024-05-08T22:29:20.437 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T22:29:20.437 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T22:29:20.437 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T22:29:20.437 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.num_levels: 7 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:29:20.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:29:20.439 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T22:29:20.439 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T22:29:20.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T22:29:20.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T22:29:20.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T22:29:20.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T22:29:20.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T22:29:20.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T22:29:20.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T22:29:20.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T22:29:20.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T22:29:20.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T22:29:20.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T22:29:20.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T22:29:20.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T22:29:20.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T22:29:20.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T22:29:20.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.table_properties_collectors: 2024-05-08T22:29:20.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.inplace_update_support: 0 2024-05-08T22:29:20.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T22:29:20.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T22:29:20.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T22:29:20.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T22:29:20.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.bloom_locality: 0 2024-05-08T22:29:20.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.max_successive_merges: 0 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.ttl: 2592000 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.enable_blob_files: false 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.min_blob_size: 0 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T22:29:20.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d0092312-2180-4a2c-9495-2ecb00be27d2 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715207360173488, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715207360175004, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94257, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 310, "table_properties": {"data_size": 92255, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13718, "raw_average_key_size": 50, "raw_value_size": 84778, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715207360, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d0092312-2180-4a2c-9495-2ecb00be27d2", "db_session_id": "WR5ND50L0C8J4HQ37RDG", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715207360175104, "job": 1, "event": "recovery_finished"} 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T22:29:20.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562c4b652000 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: DB pointer 0x562c4b640000 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** DB Stats ** 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-05-08T22:29:20.446 INFO:journalctl@ceph.mon.a.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: L0 2/0 93.88 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 107.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: Sum 2/0 93.88 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 107.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 107.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 107.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative compaction: 0.00 GB write, 20.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval compaction: 0.00 GB write, 20.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache BinnedLRUCache@0x562c4b5b51f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T22:29:20.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: starting mon.a rank 0 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: mon.a@-1(???) e1 preinit fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: mon.a@-1(???).mds e1 new map 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: mon.a@-1(???).mds e1 print_map 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout: e1 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout: legacy client fscid: -1 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout: No filesystems configured 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-08T22:29:20.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T22:29:20.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: monmap e1: 1 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:29:20.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: fsmap 2024-05-08T22:29:20.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T22:29:20.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:20 smithi062 ceph-mon[42571]: mgrmap e1: no daemons active 2024-05-08T22:29:21.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:21 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3243045878' entity='client.admin' 2024-05-08T22:29:22.990 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.555+0000 7fb0d51c1700 1 Processor -- start 2024-05-08T22:29:22.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.555+0000 7fb0d51c1700 1 -- start start 2024-05-08T22:29:22.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.555+0000 7fb0d51c1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d0105420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:22.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.555+0000 7fb0d51c1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00747c0 con 0x7fb0d0105000 2024-05-08T22:29:22.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.555+0000 7fb0ced9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d0105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:22.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.555+0000 7fb0ced9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d0105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44256/0 (socket says 172.21.15.62:44256) 2024-05-08T22:29:22.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.555+0000 7fb0ced9d700 1 -- 172.21.15.62:0/2600944034 learned_addr learned my addr 172.21.15.62:0/2600944034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:22.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.556+0000 7fb0ced9d700 1 -- 172.21.15.62:0/2600944034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d0105960 con 0x7fb0d0105000 2024-05-08T22:29:22.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.556+0000 7fb0ced9d700 1 --2- 172.21.15.62:0/2600944034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d0105420 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00098a0 tx=0x7fb0c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75595b5e259743c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:22.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.556+0000 7fb0cdd9b700 1 -- 172.21.15.62:0/2600944034 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0c0004160 con 0x7fb0d0105000 2024-05-08T22:29:22.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.556+0000 7fb0cdd9b700 1 -- 172.21.15.62:0/2600944034 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb0c00042c0 con 0x7fb0d0105000 2024-05-08T22:29:22.991 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.556+0000 7fb0cdd9b700 1 -- 172.21.15.62:0/2600944034 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0c0011a00 con 0x7fb0d0105000 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.556+0000 7fb0d51c1700 1 -- 172.21.15.62:0/2600944034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 msgr2=0x7fb0d0105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.556+0000 7fb0d51c1700 1 --2- 172.21.15.62:0/2600944034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d0105420 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00098a0 tx=0x7fb0c0009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.556+0000 7fb0d51c1700 1 -- 172.21.15.62:0/2600944034 shutdown_connections 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.556+0000 7fb0d51c1700 1 --2- 172.21.15.62:0/2600944034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d0105420 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.556+0000 7fb0d51c1700 1 -- 172.21.15.62:0/2600944034 >> 172.21.15.62:0/2600944034 conn(0x7fb0d0100ba0 msgr2=0x7fb0d0103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.556+0000 7fb0d51c1700 1 -- 172.21.15.62:0/2600944034 shutdown_connections 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.557+0000 7fb0d51c1700 1 -- 172.21.15.62:0/2600944034 wait complete. 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.557+0000 7fb0d51c1700 1 Processor -- start 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.557+0000 7fb0d51c1700 1 -- start start 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0d51c1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0d51c1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d019cbc0 con 0x7fb0d0105000 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0ced9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:22.992 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0ced9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d019c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44268/0 (socket says 172.21.15.62:44268) 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0ced9d700 1 -- 172.21.15.62:0/3243045878 learned_addr learned my addr 172.21.15.62:0/3243045878 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0ced9d700 1 -- 172.21.15.62:0/3243045878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c0009550 con 0x7fb0d0105000 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0ced9d700 1 --2- 172.21.15.62:0/3243045878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d019c680 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009fb0 tx=0x7fb0c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0b7fff700 1 -- 172.21.15.62:0/3243045878 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0c0003a40 con 0x7fb0d0105000 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0b7fff700 1 -- 172.21.15.62:0/3243045878 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb0c0003ba0 con 0x7fb0d0105000 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0d51c1700 1 -- 172.21.15.62:0/3243045878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d019cdc0 con 0x7fb0d0105000 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0b7fff700 1 -- 172.21.15.62:0/3243045878 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0c001a750 con 0x7fb0d0105000 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.558+0000 7fb0d51c1700 1 -- 172.21.15.62:0/3243045878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d019d260 con 0x7fb0d0105000 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.559+0000 7fb0b7fff700 1 -- 172.21.15.62:0/3243045878 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb0c001ac80 con 0x7fb0d0105000 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.559+0000 7fb0b7fff700 1 -- 172.21.15.62:0/3243045878 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb0c00193c0 con 0x7fb0d0105000 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.559+0000 7fb0d51c1700 1 -- 172.21.15.62:0/3243045878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d0196380 con 0x7fb0d0105000 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.562+0000 7fb0b7fff700 1 -- 172.21.15.62:0/3243045878 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fb0c002de30 con 0x7fb0d0105000 2024-05-08T22:29:22.993 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.632+0000 7fb0d51c1700 1 -- 172.21.15.62:0/3243045878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fb0d0106a80 con 0x7fb0d0105000 2024-05-08T22:29:22.994 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.637+0000 7fb0b7fff700 1 -- 172.21.15.62:0/3243045878 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb0c0024b60 con 0x7fb0d0105000 2024-05-08T22:29:22.994 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.637+0000 7fb0d51c1700 1 -- 172.21.15.62:0/3243045878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 msgr2=0x7fb0d019c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:22.994 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.637+0000 7fb0d51c1700 1 --2- 172.21.15.62:0/3243045878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d019c680 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009fb0 tx=0x7fb0c0011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:22.994 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.638+0000 7fb0d51c1700 1 -- 172.21.15.62:0/3243045878 shutdown_connections 2024-05-08T22:29:22.994 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.638+0000 7fb0d51c1700 1 --2- 172.21.15.62:0/3243045878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d0105000 0x7fb0d019c680 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:22.994 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.638+0000 7fb0d51c1700 1 -- 172.21.15.62:0/3243045878 >> 172.21.15.62:0/3243045878 conn(0x7fb0d0100ba0 msgr2=0x7fb0d01934c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:22.994 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.638+0000 7fb0d51c1700 1 -- 172.21.15.62:0/3243045878 shutdown_connections 2024-05-08T22:29:22.994 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:20.638+0000 7fb0d51c1700 1 -- 172.21.15.62:0/3243045878 wait complete. 2024-05-08T22:29:22.994 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-08T22:29:23.056 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:29:23.057 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mgr... 2024-05-08T22:29:23.057 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-08T22:29:23.058 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-08T22:29:23.606 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.x 2024-05-08T22:29:23.606 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-08T22:29:23.702 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968.target.wants/ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@.service. 2024-05-08T22:29:23.713 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:23 smithi062 systemd[1]: Starting Ceph mgr.x for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:29:26.091 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T22:29:26.092 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-05-08T22:29:26.100 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T22:29:26.100 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-05-08T22:29:26.101 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-05-08T22:29:26.101 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T22:29:26.110 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T22:29:26.110 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-05-08T22:29:26.118 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T22:29:26.118 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-05-08T22:29:26.118 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-05-08T22:29:26.118 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-08T22:29:26.119 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr to start... 2024-05-08T22:29:26.119 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr... 2024-05-08T22:29:26.343 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:26 smithi062 bash[42808]: 948a7fb5e2f1db6c8a8e948f3563bf1715662ffdfae4f725c0953a89dcd093de 2024-05-08T22:29:26.343 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:26 smithi062 systemd[1]: Started Ceph mgr.x for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:29:26.617 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:26.342+0000 7f525106a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:29:26.618 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:26.438+0000 7f525106a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:29:26.618 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:26.529+0000 7f525106a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:29:26.618 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:26.617+0000 7f525106a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:29:27.513 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:27 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3948413767' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T22:29:27.513 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:27 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:27.317+0000 7f525106a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:29:27.571 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:27.571 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T22:29:27.572 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:29:27.573 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-08T22:29:27.573 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T22:29:27.573 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:27.573 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T22:29:27.573 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T22:29:27.573 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:27.573 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T22:29:27.573 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:27.573 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T22:29:27.573 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T22:29:27.574 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T22:29:27.574 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T22:29:27.574 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T22:29:27.574 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T22:29:27.574 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:27.574 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T22:29:27.574 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T22:29:27.574 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T22:29:27.574 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T22:29:27.574 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T22:29:27.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T22:29:27.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T22:29:27.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T22:29:27.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T22:29:27.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T22:29:27.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T22:29:27.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:29:27.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:29:27.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:27.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T22:29:27.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:27.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T22:29:16.809068+0000", 2024-05-08T22:29:27.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:29:27.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:27.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T22:29:27.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:29:27.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.111+0000 7f3a108d5700 1 Processor -- start 2024-05-08T22:29:27.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.111+0000 7f3a108d5700 1 -- start start 2024-05-08T22:29:27.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.112+0000 7f3a108d5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c0725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:27.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.112+0000 7f3a108d5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c072ba0 con 0x7f3a0c0721b0 2024-05-08T22:29:27.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.112+0000 7f3a0a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c0725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:27.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.112+0000 7f3a0a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c0725d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59356/0 (socket says 172.21.15.62:59356) 2024-05-08T22:29:27.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.112+0000 7f3a0a59c700 1 -- 172.21.15.62:0/820065667 learned_addr learned my addr 172.21.15.62:0/820065667 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:27.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.113+0000 7f3a0a59c700 1 -- 172.21.15.62:0/820065667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c10df10 con 0x7f3a0c0721b0 2024-05-08T22:29:27.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.113+0000 7f3a0a59c700 1 --2- 172.21.15.62:0/820065667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c0725d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009e80 tx=0x7f3a0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e4187bec79e352 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:27.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.113+0000 7f3a0959a700 1 -- 172.21.15.62:0/820065667 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a0400e070 con 0x7f3a0c0721b0 2024-05-08T22:29:27.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.114+0000 7f3a0959a700 1 -- 172.21.15.62:0/820065667 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a0400b8c0 con 0x7f3a0c0721b0 2024-05-08T22:29:27.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.114+0000 7f3a0959a700 1 -- 172.21.15.62:0/820065667 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a040044c0 con 0x7f3a0c0721b0 2024-05-08T22:29:27.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.114+0000 7f3a108d5700 1 -- 172.21.15.62:0/820065667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 msgr2=0x7f3a0c0725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:27.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.114+0000 7f3a108d5700 1 --2- 172.21.15.62:0/820065667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c0725d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009e80 tx=0x7f3a0400b330 comp rx=0 tx=0).stop 2024-05-08T22:29:27.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.115+0000 7f3a108d5700 1 -- 172.21.15.62:0/820065667 shutdown_connections 2024-05-08T22:29:27.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.115+0000 7f3a108d5700 1 --2- 172.21.15.62:0/820065667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c0725d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:27.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.115+0000 7f3a108d5700 1 -- 172.21.15.62:0/820065667 >> 172.21.15.62:0/820065667 conn(0x7f3a0c06d800 msgr2=0x7f3a0c06fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:27.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.115+0000 7f3a108d5700 1 -- 172.21.15.62:0/820065667 shutdown_connections 2024-05-08T22:29:27.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.115+0000 7f3a108d5700 1 -- 172.21.15.62:0/820065667 wait complete. 2024-05-08T22:29:27.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.116+0000 7f3a108d5700 1 Processor -- start 2024-05-08T22:29:27.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.116+0000 7f3a108d5700 1 -- start start 2024-05-08T22:29:27.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.116+0000 7f3a108d5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c1a9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:27.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.117+0000 7f3a108d5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c1a9760 con 0x7f3a0c0721b0 2024-05-08T22:29:27.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.117+0000 7f3a0a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c1a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:27.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.117+0000 7f3a0a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c1a9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59372/0 (socket says 172.21.15.62:59372) 2024-05-08T22:29:27.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.117+0000 7f3a0a59c700 1 -- 172.21.15.62:0/3948413767 learned_addr learned my addr 172.21.15.62:0/3948413767 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:27.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.118+0000 7f3a0a59c700 1 -- 172.21.15.62:0/3948413767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a04009b30 con 0x7f3a0c0721b0 2024-05-08T22:29:27.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.118+0000 7f3a0a59c700 1 --2- 172.21.15.62:0/3948413767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c1a9220 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a040072a0 tx=0x7f3a040093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:27.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.118+0000 7f3a037fe700 1 -- 172.21.15.62:0/3948413767 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a0400e040 con 0x7f3a0c0721b0 2024-05-08T22:29:27.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.118+0000 7f3a108d5700 1 -- 172.21.15.62:0/3948413767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c1a9960 con 0x7f3a0c0721b0 2024-05-08T22:29:27.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.118+0000 7f3a108d5700 1 -- 172.21.15.62:0/3948413767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c1a9e60 con 0x7f3a0c0721b0 2024-05-08T22:29:27.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.118+0000 7f3a037fe700 1 -- 172.21.15.62:0/3948413767 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a0400bd70 con 0x7f3a0c0721b0 2024-05-08T22:29:27.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.118+0000 7f3a037fe700 1 -- 172.21.15.62:0/3948413767 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a0401c6e0 con 0x7f3a0c0721b0 2024-05-08T22:29:27.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.120+0000 7f3a037fe700 1 -- 172.21.15.62:0/3948413767 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f3a0401c920 con 0x7f3a0c0721b0 2024-05-08T22:29:27.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.120+0000 7f3a037fe700 1 -- 172.21.15.62:0/3948413767 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3a0401b3a0 con 0x7f3a0c0721b0 2024-05-08T22:29:27.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.120+0000 7f3a108d5700 1 -- 172.21.15.62:0/3948413767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ec005320 con 0x7f3a0c0721b0 2024-05-08T22:29:27.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.124+0000 7f3a037fe700 1 -- 172.21.15.62:0/3948413767 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f3a0402a060 con 0x7f3a0c0721b0 2024-05-08T22:29:27.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.197+0000 7f3a108d5700 1 -- 172.21.15.62:0/3948413767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f39ec005190 con 0x7f3a0c0721b0 2024-05-08T22:29:27.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.198+0000 7f3a037fe700 1 -- 172.21.15.62:0/3948413767 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f3a04020070 con 0x7f3a0c0721b0 2024-05-08T22:29:27.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.198+0000 7f3a108d5700 1 -- 172.21.15.62:0/3948413767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 msgr2=0x7f3a0c1a9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:27.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.198+0000 7f3a108d5700 1 --2- 172.21.15.62:0/3948413767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c1a9220 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a040072a0 tx=0x7f3a040093c0 comp rx=0 tx=0).stop 2024-05-08T22:29:27.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.198+0000 7f3a108d5700 1 -- 172.21.15.62:0/3948413767 shutdown_connections 2024-05-08T22:29:27.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.198+0000 7f3a108d5700 1 --2- 172.21.15.62:0/3948413767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a0c0721b0 0x7f3a0c1a9220 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:27.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.198+0000 7f3a108d5700 1 -- 172.21.15.62:0/3948413767 >> 172.21.15.62:0/3948413767 conn(0x7f3a0c06d800 msgr2=0x7f3a0c06f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:27.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.199+0000 7f3a108d5700 1 -- 172.21.15.62:0/3948413767 shutdown_connections 2024-05-08T22:29:27.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:27.199+0000 7f3a108d5700 1 -- 172.21.15.62:0/3948413767 wait complete. 2024-05-08T22:29:27.581 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (1/15)... 2024-05-08T22:29:27.787 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:27 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:27.582+0000 7f525106a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:29:27.787 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:27 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:27.672+0000 7f525106a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:29:28.084 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:27 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:27.787+0000 7f525106a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:29:28.084 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:27 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:27.950+0000 7f525106a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:29:28.421 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:28 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:28.083+0000 7f525106a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:29:28.422 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:28 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:28.172+0000 7f525106a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:29:28.921 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:28 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:28.561+0000 7f525106a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:29:29.268 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:28 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:28.967+0000 7f525106a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:29:29.268 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:29 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:29.171+0000 7f525106a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:29:29.671 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:29 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:29.268+0000 7f525106a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:29:30.279 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:30 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1767912247' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T22:29:30.279 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:30 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:30.092+0000 7f525106a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:29:30.279 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:30 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:30.194+0000 7f525106a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:29:30.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:30.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:29:30.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:29:30.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T22:29:30.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T22:29:30.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T22:29:30.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T22:29:30.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:30.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T22:29:30.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:30.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T22:29:30.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T22:29:30.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T22:29:30.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T22:29:30.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T22:29:30.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T22:29:30.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:30.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T22:29:30.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T22:29:16.809068+0000", 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.919+0000 7ff5e2d2f700 1 Processor -- start 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.920+0000 7ff5e2d2f700 1 -- start start 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.920+0000 7ff5e2d2f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc1095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.920+0000 7ff5e2d2f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0747e0 con 0x7ff5dc107200 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.920+0000 7ff5e0acb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc1095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.920+0000 7ff5e0acb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc1095f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59378/0 (socket says 172.21.15.62:59378) 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.920+0000 7ff5e0acb700 1 -- 172.21.15.62:0/3207778512 learned_addr learned my addr 172.21.15.62:0/3207778512 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.920+0000 7ff5e0acb700 1 -- 172.21.15.62:0/3207778512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc109b30 con 0x7ff5dc107200 2024-05-08T22:29:30.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.920+0000 7ff5e0acb700 1 --2- 172.21.15.62:0/3207778512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc1095f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00098a0 tx=0x7ff5d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c88f912370e90e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.921+0000 7ff5db7fe700 1 -- 172.21.15.62:0/3207778512 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5d0004160 con 0x7ff5dc107200 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.921+0000 7ff5db7fe700 1 -- 172.21.15.62:0/3207778512 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff5d00042c0 con 0x7ff5dc107200 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.921+0000 7ff5db7fe700 1 -- 172.21.15.62:0/3207778512 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5d0011a00 con 0x7ff5dc107200 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.921+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/3207778512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 msgr2=0x7ff5dc1095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.921+0000 7ff5e2d2f700 1 --2- 172.21.15.62:0/3207778512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc1095f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00098a0 tx=0x7ff5d0009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.921+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/3207778512 shutdown_connections 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.921+0000 7ff5e2d2f700 1 --2- 172.21.15.62:0/3207778512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc1095f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.921+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/3207778512 >> 172.21.15.62:0/3207778512 conn(0x7ff5dc100bd0 msgr2=0x7ff5dc103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.921+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/3207778512 shutdown_connections 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.921+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/3207778512 wait complete. 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.922+0000 7ff5e2d2f700 1 Processor -- start 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.922+0000 7ff5e2d2f700 1 -- start start 2024-05-08T22:29:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.922+0000 7ff5e2d2f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc106a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.922+0000 7ff5e2d2f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc106f70 con 0x7ff5dc107200 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.922+0000 7ff5e0acb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.922+0000 7ff5e0acb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59380/0 (socket says 172.21.15.62:59380) 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.922+0000 7ff5e0acb700 1 -- 172.21.15.62:0/1767912247 learned_addr learned my addr 172.21.15.62:0/1767912247 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.922+0000 7ff5e0acb700 1 -- 172.21.15.62:0/1767912247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d0009550 con 0x7ff5dc107200 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.923+0000 7ff5e0acb700 1 --2- 172.21.15.62:0/1767912247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc106a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009130 tx=0x7ff5d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.923+0000 7ff5d9ffb700 1 -- 172.21.15.62:0/1767912247 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5d00038b0 con 0x7ff5dc107200 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.923+0000 7ff5d9ffb700 1 -- 172.21.15.62:0/1767912247 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff5d0003a10 con 0x7ff5dc107200 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.923+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/1767912247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5dc1051b0 con 0x7ff5dc107200 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.923+0000 7ff5d9ffb700 1 -- 172.21.15.62:0/1767912247 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5d0019550 con 0x7ff5dc107200 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.923+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/1767912247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc105650 con 0x7ff5dc107200 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.924+0000 7ff5d9ffb700 1 -- 172.21.15.62:0/1767912247 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff5d00197e0 con 0x7ff5dc107200 2024-05-08T22:29:30.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.924+0000 7ff5d9ffb700 1 -- 172.21.15.62:0/1767912247 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff5d002d970 con 0x7ff5dc107200 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.924+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/1767912247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5dc10a8d0 con 0x7ff5dc107200 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:29.926+0000 7ff5d9ffb700 1 -- 172.21.15.62:0/1767912247 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff5d002d440 con 0x7ff5dc107200 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:30.000+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/1767912247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff5dc105890 con 0x7ff5dc107200 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:30.001+0000 7ff5d9ffb700 1 -- 172.21.15.62:0/1767912247 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7ff5d0003b80 con 0x7ff5dc107200 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:30.002+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/1767912247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 msgr2=0x7ff5dc106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:30.002+0000 7ff5e2d2f700 1 --2- 172.21.15.62:0/1767912247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc106a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009130 tx=0x7ff5d0011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:30.002+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/1767912247 shutdown_connections 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:30.002+0000 7ff5e2d2f700 1 --2- 172.21.15.62:0/1767912247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5dc107200 0x7ff5dc106a30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:30.002+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/1767912247 >> 172.21.15.62:0/1767912247 conn(0x7ff5dc100bd0 msgr2=0x7ff5dc1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:30.002+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/1767912247 shutdown_connections 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:30.002+0000 7ff5e2d2f700 1 -- 172.21.15.62:0/1767912247 wait complete. 2024-05-08T22:29:30.383 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (2/15)... 2024-05-08T22:29:30.671 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:30 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:30.351+0000 7f525106a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:29:31.401 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:31 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:31.009+0000 7f525106a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:29:31.401 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:31 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:31.199+0000 7f525106a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:29:31.671 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:31 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:31.401+0000 7f525106a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:29:32.671 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:32 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:32.374+0000 7f525106a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:29:32.966 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:32 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:32.676+0000 7f525106a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:29:32.967 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:32 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:32.767+0000 7f525106a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:29:33.340 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:32 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:32.966+0000 7f525106a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: Activating manager daemon x 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: mgrmap e2: x(active, starting, since 0.00328512s) 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: Manager daemon x is now available 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' 2024-05-08T22:29:33.341 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' 2024-05-08T22:29:33.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:33 smithi062 ceph-mon[42571]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' 2024-05-08T22:29:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:29:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:29:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T22:29:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T22:29:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T22:29:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T22:29:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T22:29:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T22:29:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T22:29:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T22:29:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T22:29:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T22:29:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T22:29:16.809068+0000", 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.905+0000 7f09744d9700 1 Processor -- start 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.905+0000 7f09744d9700 1 -- start start 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.905+0000 7f09744d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c1095b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.905+0000 7f09744d9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c074820 con 0x7f096c1071c0 2024-05-08T22:29:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.906+0000 7f0972275700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c1095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.906+0000 7f0972275700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c1095b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59440/0 (socket says 172.21.15.62:59440) 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.906+0000 7f0972275700 1 -- 172.21.15.62:0/2521895503 learned_addr learned my addr 172.21.15.62:0/2521895503 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.906+0000 7f0972275700 1 -- 172.21.15.62:0/2521895503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c109af0 con 0x7f096c1071c0 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.906+0000 7f0972275700 1 --2- 172.21.15.62:0/2521895503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c1095b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f095c0098a0 tx=0x7f095c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff499630f3245529 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.906+0000 7f0971273700 1 -- 172.21.15.62:0/2521895503 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c004160 con 0x7f096c1071c0 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.906+0000 7f0971273700 1 -- 172.21.15.62:0/2521895503 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f095c0042c0 con 0x7f096c1071c0 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.906+0000 7f0971273700 1 -- 172.21.15.62:0/2521895503 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c011a00 con 0x7f096c1071c0 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.907+0000 7f09744d9700 1 -- 172.21.15.62:0/2521895503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 msgr2=0x7f096c1095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.907+0000 7f09744d9700 1 --2- 172.21.15.62:0/2521895503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c1095b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f095c0098a0 tx=0x7f095c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.907+0000 7f09744d9700 1 -- 172.21.15.62:0/2521895503 shutdown_connections 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.907+0000 7f09744d9700 1 --2- 172.21.15.62:0/2521895503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c1095b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.907+0000 7f09744d9700 1 -- 172.21.15.62:0/2521895503 >> 172.21.15.62:0/2521895503 conn(0x7f096c100bd0 msgr2=0x7f096c103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.907+0000 7f09744d9700 1 -- 172.21.15.62:0/2521895503 shutdown_connections 2024-05-08T22:29:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.907+0000 7f09744d9700 1 -- 172.21.15.62:0/2521895503 wait complete. 2024-05-08T22:29:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.908+0000 7f09744d9700 1 Processor -- start 2024-05-08T22:29:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.908+0000 7f09744d9700 1 -- start start 2024-05-08T22:29:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.908+0000 7f09744d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.908+0000 7f09744d9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c19aa00 con 0x7f096c1071c0 2024-05-08T22:29:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.908+0000 7f0972275700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.908+0000 7f0972275700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59448/0 (socket says 172.21.15.62:59448) 2024-05-08T22:29:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.908+0000 7f0972275700 1 -- 172.21.15.62:0/1820659283 learned_addr learned my addr 172.21.15.62:0/1820659283 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.909+0000 7f0972275700 1 -- 172.21.15.62:0/1820659283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c009550 con 0x7f096c1071c0 2024-05-08T22:29:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.909+0000 7f0972275700 1 --2- 172.21.15.62:0/1820659283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c19a4c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f095c009130 tx=0x7f095c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.909+0000 7f09637fe700 1 -- 172.21.15.62:0/1820659283 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c0038b0 con 0x7f096c1071c0 2024-05-08T22:29:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.909+0000 7f09637fe700 1 -- 172.21.15.62:0/1820659283 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f095c003a10 con 0x7f096c1071c0 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.909+0000 7f09637fe700 1 -- 172.21.15.62:0/1820659283 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c019550 con 0x7f096c1071c0 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.909+0000 7f09744d9700 1 -- 172.21.15.62:0/1820659283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c19ac00 con 0x7f096c1071c0 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.909+0000 7f09744d9700 1 -- 172.21.15.62:0/1820659283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c19b0a0 con 0x7f096c1071c0 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.910+0000 7f09637fe700 1 -- 172.21.15.62:0/1820659283 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 2) v1 ==== 45297+0+0 (secure 0 0 0) 0x7f095c01a430 con 0x7f096c1071c0 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.910+0000 7f09744d9700 1 -- 172.21.15.62:0/1820659283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c10a8f0 con 0x7f096c1071c0 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.910+0000 7f09637fe700 1 -- 172.21.15.62:0/1820659283 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f095c04c040 con 0x7f096c1071c0 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.912+0000 7f09637fe700 1 -- 172.21.15.62:0/1820659283 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f095c003b80 con 0x7f096c1071c0 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.983+0000 7f09637fe700 1 -- 172.21.15.62:0/1820659283 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7f095c019900 con 0x7f096c1071c0 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.983+0000 7f09637fe700 1 --2- 172.21.15.62:0/1820659283 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f0958039290 0x7f095803b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.984+0000 7f0971a74700 1 --2- 172.21.15.62:0/1820659283 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f0958039290 0x7f095803b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.985+0000 7f0971a74700 1 --2- 172.21.15.62:0/1820659283 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f0958039290 0x7f095803b680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0968006fd0 tx=0x7f0968006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.989+0000 7f09744d9700 1 -- 172.21.15.62:0/1820659283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f096c10ab60 con 0x7f096c1071c0 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.989+0000 7f09637fe700 1 -- 172.21.15.62:0/1820659283 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f095c02dc60 con 0x7f096c1071c0 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.990+0000 7f09744d9700 1 -- 172.21.15.62:0/1820659283 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f0958039290 msgr2=0x7f095803b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.990+0000 7f09744d9700 1 --2- 172.21.15.62:0/1820659283 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f0958039290 0x7f095803b680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0968006fd0 tx=0x7f0968006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.990+0000 7f09744d9700 1 -- 172.21.15.62:0/1820659283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 msgr2=0x7f096c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.990+0000 7f09744d9700 1 --2- 172.21.15.62:0/1820659283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c19a4c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f095c009130 tx=0x7f095c011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.990+0000 7f09744d9700 1 -- 172.21.15.62:0/1820659283 shutdown_connections 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.990+0000 7f09744d9700 1 --2- 172.21.15.62:0/1820659283 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f0958039290 0x7f095803b680 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.990+0000 7f09744d9700 1 --2- 172.21.15.62:0/1820659283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096c1071c0 0x7f096c19a4c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.990+0000 7f09744d9700 1 -- 172.21.15.62:0/1820659283 >> 172.21.15.62:0/1820659283 conn(0x7f096c100bd0 msgr2=0x7f096c1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.990+0000 7f09744d9700 1 -- 172.21.15.62:0/1820659283 shutdown_connections 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:33.990+0000 7f09744d9700 1 -- 172.21.15.62:0/1820659283 wait complete. 2024-05-08T22:29:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:mgr is available 2024-05-08T22:29:35.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:34 smithi062 ceph-mon[42571]: mgrmap e3: x(active, since 1.00611s) 2024-05-08T22:29:35.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:34 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1820659283' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T22:29:35.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:34 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/4273590492' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T22:29:35.308 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:35.308 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T22:29:35.308 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T22:29:35.309 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.727+0000 7fb462e29700 1 Processor -- start 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.727+0000 7fb462e29700 1 -- start start 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.727+0000 7fb462e29700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c106b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.727+0000 7fb462e29700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c107140 con 0x7fb45c106750 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.727+0000 7fb460bc5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c106b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.727+0000 7fb460bc5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c106b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59456/0 (socket says 172.21.15.62:59456) 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.727+0000 7fb460bc5700 1 -- 172.21.15.62:0/3267635188 learned_addr learned my addr 172.21.15.62:0/3267635188 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.727+0000 7fb460bc5700 1 -- 172.21.15.62:0/3267635188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c107950 con 0x7fb45c106750 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.728+0000 7fb460bc5700 1 --2- 172.21.15.62:0/3267635188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c106b70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4500098a0 tx=0x7fb450009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4bd445398cc7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.728+0000 7fb45b7fe700 1 -- 172.21.15.62:0/3267635188 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb450004160 con 0x7fb45c106750 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.728+0000 7fb45b7fe700 1 -- 172.21.15.62:0/3267635188 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4500042c0 con 0x7fb45c106750 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.728+0000 7fb45b7fe700 1 -- 172.21.15.62:0/3267635188 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb450011a00 con 0x7fb45c106750 2024-05-08T22:29:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.728+0000 7fb462e29700 1 -- 172.21.15.62:0/3267635188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 msgr2=0x7fb45c106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.728+0000 7fb462e29700 1 --2- 172.21.15.62:0/3267635188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c106b70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4500098a0 tx=0x7fb450009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.728+0000 7fb462e29700 1 -- 172.21.15.62:0/3267635188 shutdown_connections 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.728+0000 7fb462e29700 1 --2- 172.21.15.62:0/3267635188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c106b70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.728+0000 7fb462e29700 1 -- 172.21.15.62:0/3267635188 >> 172.21.15.62:0/3267635188 conn(0x7fb45c101cf0 msgr2=0x7fb45c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.728+0000 7fb462e29700 1 -- 172.21.15.62:0/3267635188 shutdown_connections 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.728+0000 7fb462e29700 1 -- 172.21.15.62:0/3267635188 wait complete. 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.729+0000 7fb462e29700 1 Processor -- start 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.729+0000 7fb462e29700 1 -- start start 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.729+0000 7fb462e29700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.730+0000 7fb462e29700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c19aa20 con 0x7fb45c106750 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.730+0000 7fb460bc5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.730+0000 7fb460bc5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59466/0 (socket says 172.21.15.62:59466) 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.730+0000 7fb460bc5700 1 -- 172.21.15.62:0/4273590492 learned_addr learned my addr 172.21.15.62:0/4273590492 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:35.311 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.730+0000 7fb460bc5700 1 -- 172.21.15.62:0/4273590492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb450009550 con 0x7fb45c106750 2024-05-08T22:29:35.312 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.730+0000 7fb460bc5700 1 --2- 172.21.15.62:0/4273590492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c19a4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb450009130 tx=0x7fb450011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:35.312 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.730+0000 7fb459ffb700 1 -- 172.21.15.62:0/4273590492 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4500038b0 con 0x7fb45c106750 2024-05-08T22:29:35.312 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.730+0000 7fb459ffb700 1 -- 172.21.15.62:0/4273590492 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb450003a10 con 0x7fb45c106750 2024-05-08T22:29:35.312 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.730+0000 7fb459ffb700 1 -- 172.21.15.62:0/4273590492 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb450019550 con 0x7fb45c106750 2024-05-08T22:29:35.312 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.730+0000 7fb462e29700 1 -- 172.21.15.62:0/4273590492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c19ac20 con 0x7fb45c106750 2024-05-08T22:29:35.312 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.730+0000 7fb462e29700 1 -- 172.21.15.62:0/4273590492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c19b0c0 con 0x7fb45c106750 2024-05-08T22:29:35.312 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.731+0000 7fb459ffb700 1 -- 172.21.15.62:0/4273590492 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7fb45001a960 con 0x7fb45c106750 2024-05-08T22:29:35.312 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.731+0000 7fb459ffb700 1 --2- 172.21.15.62:0/4273590492 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7fb44c038570 0x7fb44c03aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:35.312 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.731+0000 7fb462e29700 1 -- 172.21.15.62:0/4273590492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45c10ab60 con 0x7fb45c106750 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.731+0000 7fb459ffb700 1 -- 172.21.15.62:0/4273590492 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb45004c150 con 0x7fb45c106750 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.732+0000 7fb45bfff700 1 --2- 172.21.15.62:0/4273590492 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7fb44c038570 0x7fb44c03aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.732+0000 7fb45bfff700 1 --2- 172.21.15.62:0/4273590492 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7fb44c038570 0x7fb44c03aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb448006fd0 tx=0x7fb448006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.737+0000 7fb459ffb700 1 -- 172.21.15.62:0/4273590492 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb4500197c0 con 0x7fb45c106750 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.936+0000 7fb462e29700 1 -- 172.21.15.62:0/4273590492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fb45c19d950 con 0x7fb45c106750 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.937+0000 7fb459ffb700 1 -- 172.21.15.62:0/4273590492 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fb450023b40 con 0x7fb45c106750 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.939+0000 7fb462e29700 1 -- 172.21.15.62:0/4273590492 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7fb44c038570 msgr2=0x7fb44c03aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.939+0000 7fb462e29700 1 --2- 172.21.15.62:0/4273590492 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7fb44c038570 0x7fb44c03aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb448006fd0 tx=0x7fb448006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.939+0000 7fb462e29700 1 -- 172.21.15.62:0/4273590492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 msgr2=0x7fb45c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.939+0000 7fb462e29700 1 --2- 172.21.15.62:0/4273590492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c19a4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb450009130 tx=0x7fb450011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.939+0000 7fb462e29700 1 -- 172.21.15.62:0/4273590492 shutdown_connections 2024-05-08T22:29:35.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.939+0000 7fb462e29700 1 --2- 172.21.15.62:0/4273590492 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7fb44c038570 0x7fb44c03aa30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:35.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.939+0000 7fb462e29700 1 --2- 172.21.15.62:0/4273590492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb45c106750 0x7fb45c19a4e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:35.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.939+0000 7fb462e29700 1 -- 172.21.15.62:0/4273590492 >> 172.21.15.62:0/4273590492 conn(0x7fb45c101cf0 msgr2=0x7fb45c1028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:35.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.939+0000 7fb462e29700 1 -- 172.21.15.62:0/4273590492 shutdown_connections 2024-05-08T22:29:35.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:34.939+0000 7fb462e29700 1 -- 172.21.15.62:0/4273590492 wait complete. 2024-05-08T22:29:35.315 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling cephadm module... 2024-05-08T22:29:36.311 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:35 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2205659201' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T22:29:36.312 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:36 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: ignoring --setuser ceph since I am not root 2024-05-08T22:29:36.312 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:36 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: ignoring --setgroup ceph since I am not root 2024-05-08T22:29:36.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.636+0000 7f47e38ee700 1 Processor -- start 2024-05-08T22:29:36.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.636+0000 7f47e38ee700 1 -- start start 2024-05-08T22:29:36.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.636+0000 7f47e38ee700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc109580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:36.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.636+0000 7f47e38ee700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc079eb0 con 0x7f47dc107190 2024-05-08T22:29:36.389 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.636+0000 7f47e168a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc109580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.636+0000 7f47e168a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc109580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59476/0 (socket says 172.21.15.62:59476) 2024-05-08T22:29:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.636+0000 7f47e168a700 1 -- 172.21.15.62:0/770917697 learned_addr learned my addr 172.21.15.62:0/770917697 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.636+0000 7f47e168a700 1 -- 172.21.15.62:0/770917697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc109ac0 con 0x7f47dc107190 2024-05-08T22:29:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.637+0000 7f47e168a700 1 --2- 172.21.15.62:0/770917697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc109580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f47d80098a0 tx=0x7f47d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917e9291d39974fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.637+0000 7f47d3fff700 1 -- 172.21.15.62:0/770917697 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47d8004160 con 0x7f47dc107190 2024-05-08T22:29:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.637+0000 7f47d3fff700 1 -- 172.21.15.62:0/770917697 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f47d80042c0 con 0x7f47dc107190 2024-05-08T22:29:36.390 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.637+0000 7f47d3fff700 1 -- 172.21.15.62:0/770917697 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47d8011a00 con 0x7f47dc107190 2024-05-08T22:29:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.637+0000 7f47e38ee700 1 -- 172.21.15.62:0/770917697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 msgr2=0x7f47dc109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.637+0000 7f47e38ee700 1 --2- 172.21.15.62:0/770917697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc109580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f47d80098a0 tx=0x7f47d8009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.637+0000 7f47e38ee700 1 -- 172.21.15.62:0/770917697 shutdown_connections 2024-05-08T22:29:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.637+0000 7f47e38ee700 1 --2- 172.21.15.62:0/770917697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc109580 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.637+0000 7f47e38ee700 1 -- 172.21.15.62:0/770917697 >> 172.21.15.62:0/770917697 conn(0x7f47dc100c40 msgr2=0x7f47dc103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.637+0000 7f47e38ee700 1 -- 172.21.15.62:0/770917697 shutdown_connections 2024-05-08T22:29:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.638+0000 7f47e38ee700 1 -- 172.21.15.62:0/770917697 wait complete. 2024-05-08T22:29:36.391 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.638+0000 7f47e38ee700 1 Processor -- start 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.638+0000 7f47e38ee700 1 -- start start 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.638+0000 7f47e38ee700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.638+0000 7f47e38ee700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc19a950 con 0x7f47dc107190 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.638+0000 7f47e168a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.638+0000 7f47e168a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59478/0 (socket says 172.21.15.62:59478) 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.638+0000 7f47e168a700 1 -- 172.21.15.62:0/2205659201 learned_addr learned my addr 172.21.15.62:0/2205659201 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.639+0000 7f47e168a700 1 -- 172.21.15.62:0/2205659201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d8009550 con 0x7f47dc107190 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.639+0000 7f47e168a700 1 --2- 172.21.15.62:0/2205659201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc19a410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009fb0 tx=0x7f47d8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.639+0000 7f47d27fc700 1 -- 172.21.15.62:0/2205659201 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47d8003a40 con 0x7f47dc107190 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.639+0000 7f47d27fc700 1 -- 172.21.15.62:0/2205659201 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f47d8003ba0 con 0x7f47dc107190 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.639+0000 7f47e38ee700 1 -- 172.21.15.62:0/2205659201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc19ab50 con 0x7f47dc107190 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.639+0000 7f47d27fc700 1 -- 172.21.15.62:0/2205659201 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47d801a750 con 0x7f47dc107190 2024-05-08T22:29:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.639+0000 7f47e38ee700 1 -- 172.21.15.62:0/2205659201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc19aff0 con 0x7f47dc107190 2024-05-08T22:29:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.640+0000 7f47d27fc700 1 -- 172.21.15.62:0/2205659201 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7f47d801a980 con 0x7f47dc107190 2024-05-08T22:29:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.640+0000 7f47d27fc700 1 --2- 172.21.15.62:0/2205659201 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f47c8038590 0x7f47c803aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.640+0000 7f47e38ee700 1 -- 172.21.15.62:0/2205659201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc10a8c0 con 0x7f47dc107190 2024-05-08T22:29:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.640+0000 7f47d27fc700 1 -- 172.21.15.62:0/2205659201 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f47d804c370 con 0x7f47dc107190 2024-05-08T22:29:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.640+0000 7f47e0e89700 1 --2- 172.21.15.62:0/2205659201 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f47c8038590 0x7f47c803aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.641+0000 7f47e0e89700 1 --2- 172.21.15.62:0/2205659201 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f47c8038590 0x7f47c803aa50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006fd0 tx=0x7f47cc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.645+0000 7f47d27fc700 1 -- 172.21.15.62:0/2205659201 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f47d801ac30 con 0x7f47dc107190 2024-05-08T22:29:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.904+0000 7f47e38ee700 1 -- 172.21.15.62:0/2205659201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f47dc10ab00 con 0x7f47dc107190 2024-05-08T22:29:36.393 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.984+0000 7f47d27fc700 1 -- 172.21.15.62:0/2205659201 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 4) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f47d802d910 con 0x7f47dc107190 2024-05-08T22:29:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.985+0000 7f47d27fc700 1 -- 172.21.15.62:0/2205659201 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v4) v1 ==== 86+0+0 (secure 0 0 0) 0x7f47d801ac30 con 0x7f47dc107190 2024-05-08T22:29:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.988+0000 7f47e38ee700 1 -- 172.21.15.62:0/2205659201 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f47c8038590 msgr2=0x7f47c803aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.988+0000 7f47e38ee700 1 --2- 172.21.15.62:0/2205659201 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f47c8038590 0x7f47c803aa50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006fd0 tx=0x7f47cc006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.988+0000 7f47e38ee700 1 -- 172.21.15.62:0/2205659201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 msgr2=0x7f47dc19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.988+0000 7f47e38ee700 1 --2- 172.21.15.62:0/2205659201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc19a410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009fb0 tx=0x7f47d8011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.988+0000 7f47e38ee700 1 -- 172.21.15.62:0/2205659201 shutdown_connections 2024-05-08T22:29:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.988+0000 7f47e38ee700 1 --2- 172.21.15.62:0/2205659201 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f47c8038590 0x7f47c803aa50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.988+0000 7f47e38ee700 1 --2- 172.21.15.62:0/2205659201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47dc107190 0x7f47dc19a410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.988+0000 7f47e38ee700 1 -- 172.21.15.62:0/2205659201 >> 172.21.15.62:0/2205659201 conn(0x7f47dc100c40 msgr2=0x7f47dc1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.988+0000 7f47e38ee700 1 -- 172.21.15.62:0/2205659201 shutdown_connections 2024-05-08T22:29:36.394 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:35.988+0000 7f47e38ee700 1 -- 172.21.15.62:0/2205659201 wait complete. 2024-05-08T22:29:36.631 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:36 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:36.437+0000 7fa26aa24200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:29:36.631 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:36 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:36.539+0000 7fa26aa24200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:29:36.921 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:36 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:36.631+0000 7fa26aa24200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:29:36.922 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:36 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:36.720+0000 7fa26aa24200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:29:37.328 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:36 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2205659201' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T22:29:37.328 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:36 smithi062 ceph-mon[42571]: mgrmap e4: x(active, since 3s) 2024-05-08T22:29:37.401 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:29:37.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 4, 2024-05-08T22:29:37.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T22:29:37.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T22:29:37.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T22:29:37.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:29:37.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.771+0000 7f4f527b4700 1 Processor -- start 2024-05-08T22:29:37.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.771+0000 7f4f527b4700 1 -- start start 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.771+0000 7f4f527b4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c109590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.771+0000 7f4f527b4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c079eb0 con 0x7f4f4c1071a0 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.771+0000 7f4f4bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c109590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.771+0000 7f4f4bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c109590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59594/0 (socket says 172.21.15.62:59594) 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.771+0000 7f4f4bfff700 1 -- 172.21.15.62:0/3203082157 learned_addr learned my addr 172.21.15.62:0/3203082157 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.771+0000 7f4f4bfff700 1 -- 172.21.15.62:0/3203082157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c109ad0 con 0x7f4f4c1071a0 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.772+0000 7f4f4bfff700 1 --2- 172.21.15.62:0/3203082157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c109590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0098a0 tx=0x7f4f3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7ec3f218ea2df8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.772+0000 7f4f4affd700 1 -- 172.21.15.62:0/3203082157 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f3c004160 con 0x7f4f4c1071a0 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.772+0000 7f4f4affd700 1 -- 172.21.15.62:0/3203082157 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4f3c0042c0 con 0x7f4f4c1071a0 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.772+0000 7f4f4affd700 1 -- 172.21.15.62:0/3203082157 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f3c011a00 con 0x7f4f4c1071a0 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.772+0000 7f4f527b4700 1 -- 172.21.15.62:0/3203082157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 msgr2=0x7f4f4c109590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:37.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.772+0000 7f4f527b4700 1 --2- 172.21.15.62:0/3203082157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c109590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0098a0 tx=0x7f4f3c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.773+0000 7f4f527b4700 1 -- 172.21.15.62:0/3203082157 shutdown_connections 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.773+0000 7f4f527b4700 1 --2- 172.21.15.62:0/3203082157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c109590 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.773+0000 7f4f527b4700 1 -- 172.21.15.62:0/3203082157 >> 172.21.15.62:0/3203082157 conn(0x7f4f4c100bb0 msgr2=0x7f4f4c103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.773+0000 7f4f527b4700 1 -- 172.21.15.62:0/3203082157 shutdown_connections 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.773+0000 7f4f527b4700 1 -- 172.21.15.62:0/3203082157 wait complete. 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.773+0000 7f4f527b4700 1 Processor -- start 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.773+0000 7f4f527b4700 1 -- start start 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.774+0000 7f4f527b4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.774+0000 7f4f527b4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c19a960 con 0x7f4f4c1071a0 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.774+0000 7f4f4bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.774+0000 7f4f4bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59596/0 (socket says 172.21.15.62:59596) 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.774+0000 7f4f4bfff700 1 -- 172.21.15.62:0/2986721068 learned_addr learned my addr 172.21.15.62:0/2986721068 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.774+0000 7f4f4bfff700 1 -- 172.21.15.62:0/2986721068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c009550 con 0x7f4f4c1071a0 2024-05-08T22:29:37.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.774+0000 7f4f4bfff700 1 --2- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c19a420 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009fb0 tx=0x7f4f3c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.774+0000 7f4f497fa700 1 -- 172.21.15.62:0/2986721068 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f3c003a40 con 0x7f4f4c1071a0 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.774+0000 7f4f527b4700 1 -- 172.21.15.62:0/2986721068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4c19ab60 con 0x7f4f4c1071a0 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.774+0000 7f4f497fa700 1 -- 172.21.15.62:0/2986721068 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4f3c003ba0 con 0x7f4f4c1071a0 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.774+0000 7f4f497fa700 1 -- 172.21.15.62:0/2986721068 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f3c01a750 con 0x7f4f4c1071a0 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.775+0000 7f4f527b4700 1 -- 172.21.15.62:0/2986721068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c19b000 con 0x7f4f4c1071a0 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.775+0000 7f4f497fa700 1 -- 172.21.15.62:0/2986721068 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f4f3c01a9d0 con 0x7f4f4c1071a0 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.775+0000 7f4f527b4700 1 -- 172.21.15.62:0/2986721068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4c10a8d0 con 0x7f4f4c1071a0 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.776+0000 7f4f497fa700 1 --2- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f4f340386b0 0x7f4f3403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.776+0000 7f4f497fa700 1 -- 172.21.15.62:0/2986721068 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4f3c04c410 con 0x7f4f4c1071a0 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.776+0000 7f4f4b7fe700 1 -- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f4f340386b0 msgr2=0x7f4f3403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3409257313 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.776+0000 7f4f4b7fe700 1 --2- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f4f340386b0 0x7f4f3403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.781+0000 7f4f497fa700 1 -- 172.21.15.62:0/2986721068 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4f3c018d20 con 0x7f4f4c1071a0 2024-05-08T22:29:37.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.977+0000 7f4f4b7fe700 1 -- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f4f340386b0 msgr2=0x7f4f3403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3409257313 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:36.977+0000 7f4f4b7fe700 1 --2- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f4f340386b0 0x7f4f3403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.041+0000 7f4f527b4700 1 -- 172.21.15.62:0/2986721068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f4f4c19d890 con 0x7f4f4c1071a0 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.041+0000 7f4f497fa700 1 -- 172.21.15.62:0/2986721068 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v4) v1 ==== 56+0+88 (secure 0 0 0) 0x7f4f3c018b40 con 0x7f4f4c1071a0 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.044+0000 7f4f527b4700 1 -- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f4f340386b0 msgr2=0x7f4f3403ab70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.044+0000 7f4f527b4700 1 --2- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f4f340386b0 0x7f4f3403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.044+0000 7f4f527b4700 1 -- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 msgr2=0x7f4f4c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.044+0000 7f4f527b4700 1 --2- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c19a420 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009fb0 tx=0x7f4f3c011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.044+0000 7f4f527b4700 1 -- 172.21.15.62:0/2986721068 shutdown_connections 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.044+0000 7f4f527b4700 1 --2- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f4f340386b0 0x7f4f3403ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.044+0000 7f4f527b4700 1 --2- 172.21.15.62:0/2986721068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f4c1071a0 0x7f4f4c19a420 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.044+0000 7f4f527b4700 1 -- 172.21.15.62:0/2986721068 >> 172.21.15.62:0/2986721068 conn(0x7f4f4c100bb0 msgr2=0x7f4f4c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.044+0000 7f4f527b4700 1 -- 172.21.15.62:0/2986721068 shutdown_connections 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.044+0000 7f4f527b4700 1 -- 172.21.15.62:0/2986721068 wait complete. 2024-05-08T22:29:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2024-05-08T22:29:37.413 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 4... 2024-05-08T22:29:37.670 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:37 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:37.396+0000 7fa26aa24200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:29:37.921 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:37 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:37.670+0000 7fa26aa24200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:29:37.922 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:37 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:37.764+0000 7fa26aa24200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:29:37.922 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:37 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:37.879+0000 7fa26aa24200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:29:38.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:37 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2986721068' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T22:29:38.266 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:38 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:38.043+0000 7fa26aa24200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:29:38.266 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:38 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:38.176+0000 7fa26aa24200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:29:38.651 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:38 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:38.265+0000 7fa26aa24200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:29:38.921 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:38 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:38.651+0000 7fa26aa24200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:29:39.360 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:39 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:39.055+0000 7fa26aa24200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:29:39.360 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:39 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:39.261+0000 7fa26aa24200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:29:39.671 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:39 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:39.359+0000 7fa26aa24200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:29:40.422 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:40 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:40.161+0000 7fa26aa24200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:29:40.422 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:40 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:40.263+0000 7fa26aa24200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:29:40.671 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:40 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:40.420+0000 7fa26aa24200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:29:41.421 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:41 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:41.080+0000 7fa26aa24200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:29:41.421 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:41 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:41.270+0000 7fa26aa24200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:29:41.921 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:41 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:41.473+0000 7fa26aa24200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:29:42.747 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:42 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:42.447+0000 7fa26aa24200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:29:43.043 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:42 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:42.747+0000 7fa26aa24200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:29:43.044 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:42 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:42.841+0000 7fa26aa24200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:29:43.421 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:43 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:43.043+0000 7fa26aa24200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:29:43.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: Active manager daemon x restarted 2024-05-08T22:29:43.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: Activating manager daemon x 2024-05-08T22:29:43.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T22:29:43.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: mgrmap e5: x(active, starting, since 0.00345048s) 2024-05-08T22:29:43.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:29:43.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:29:43.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:29:43.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:29:43.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:29:43.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: Manager daemon x is now available 2024-05-08T22:29:43.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:43.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:43 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:44.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:44 smithi062 ceph-mon[42571]: Found migration_current of "None". Setting to last migration. 2024-05-08T22:29:44.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:44 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:29:44.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:44 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:29:44.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:44 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:29:44.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:44 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:29:44.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:44 smithi062 ceph-mon[42571]: mgrmap e6: x(active, since 1.00578s) 2024-05-08T22:29:44.432 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 6, 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.745+0000 7f323a992700 1 Processor -- start 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.745+0000 7f323a992700 1 -- start start 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.745+0000 7f323a992700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f32341095b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.745+0000 7f323a992700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3234074820 con 0x7f32341071c0 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.745+0000 7f3233fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f32341095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.745+0000 7f3233fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f32341095b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59598/0 (socket says 172.21.15.62:59598) 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.745+0000 7f3233fff700 1 -- 172.21.15.62:0/2985000783 learned_addr learned my addr 172.21.15.62:0/2985000783 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.745+0000 7f3233fff700 1 -- 172.21.15.62:0/2985000783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3234109af0 con 0x7f32341071c0 2024-05-08T22:29:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.745+0000 7f3233fff700 1 --2- 172.21.15.62:0/2985000783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f32341095b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f32240098a0 tx=0x7f3224009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a2f1d54adb8ac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.746+0000 7f32337fe700 1 -- 172.21.15.62:0/2985000783 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3224004160 con 0x7f32341071c0 2024-05-08T22:29:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.746+0000 7f32337fe700 1 -- 172.21.15.62:0/2985000783 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f32240042c0 con 0x7f32341071c0 2024-05-08T22:29:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.746+0000 7f32337fe700 1 -- 172.21.15.62:0/2985000783 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3224011a00 con 0x7f32341071c0 2024-05-08T22:29:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.746+0000 7f323a992700 1 -- 172.21.15.62:0/2985000783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 msgr2=0x7f32341095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.746+0000 7f323a992700 1 --2- 172.21.15.62:0/2985000783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f32341095b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f32240098a0 tx=0x7f3224009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.746+0000 7f323a992700 1 -- 172.21.15.62:0/2985000783 shutdown_connections 2024-05-08T22:29:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.746+0000 7f323a992700 1 --2- 172.21.15.62:0/2985000783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f32341095b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.746+0000 7f323a992700 1 -- 172.21.15.62:0/2985000783 >> 172.21.15.62:0/2985000783 conn(0x7f3234100bd0 msgr2=0x7f3234103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.746+0000 7f323a992700 1 -- 172.21.15.62:0/2985000783 shutdown_connections 2024-05-08T22:29:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.746+0000 7f323a992700 1 -- 172.21.15.62:0/2985000783 wait complete. 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.747+0000 7f323a992700 1 Processor -- start 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.747+0000 7f323a992700 1 -- start start 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.747+0000 7f323a992700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f323419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.747+0000 7f323a992700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323419cc00 con 0x7f32341071c0 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.747+0000 7f3233fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f323419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.747+0000 7f3233fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f323419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59610/0 (socket says 172.21.15.62:59610) 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.747+0000 7f3233fff700 1 -- 172.21.15.62:0/2318171923 learned_addr learned my addr 172.21.15.62:0/2318171923 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.748+0000 7f3233fff700 1 -- 172.21.15.62:0/2318171923 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3224009550 con 0x7f32341071c0 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.748+0000 7f3233fff700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f323419c6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3224009130 tx=0x7f3224011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.748+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32240038b0 con 0x7f32341071c0 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.748+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3224003a10 con 0x7f32341071c0 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.748+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3224019550 con 0x7f32341071c0 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.748+0000 7f323a992700 1 -- 172.21.15.62:0/2318171923 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323419ce00 con 0x7f32341071c0 2024-05-08T22:29:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.748+0000 7f323a992700 1 -- 172.21.15.62:0/2318171923 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323419d2a0 con 0x7f32341071c0 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.749+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f322401a960 con 0x7f32341071c0 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.749+0000 7f3231ffb700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 0x7f322003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.749+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 --> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f322003b2b0 con 0x7f32200386e0 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.749+0000 7f322bfff700 1 -- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 msgr2=0x7f322003aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3409257313 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.749+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f322404c400 con 0x7f32341071c0 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.749+0000 7f322bfff700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 0x7f322003aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.950+0000 7f322bfff700 1 -- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 msgr2=0x7f322003aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3409257313 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:37.950+0000 7f322bfff700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 0x7f322003aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:38.351+0000 7f322bfff700 1 -- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 msgr2=0x7f322003aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3409257313 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:38.351+0000 7f322bfff700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 0x7f322003aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:39.152+0000 7f322bfff700 1 -- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 msgr2=0x7f322003aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3409257313 2024-05-08T22:29:44.436 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:39.152+0000 7f322bfff700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 0x7f322003aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T22:29:44.437 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:40.753+0000 7f322bfff700 1 -- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 msgr2=0x7f322003aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3409257313 2024-05-08T22:29:44.437 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:40.753+0000 7f322bfff700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 0x7f322003aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T22:29:44.437 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:43.059+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 5) v1 ==== 45308+0+0 (secure 0 0 0) 0x7f3224019ce0 con 0x7f32341071c0 2024-05-08T22:29:44.437 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:43.059+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 msgr2=0x7f322003aba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:29:44.437 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:43.059+0000 7f3231ffb700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 0x7f322003aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:44.437 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.060+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 6) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f322402ad30 con 0x7f32341071c0 2024-05-08T22:29:44.437 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.060+0000 7f3231ffb700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f322003bfc0 0x7f322003e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:44.437 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.061+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 --> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f322003b2b0 con 0x7f322003bfc0 2024-05-08T22:29:44.437 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.061+0000 7f322bfff700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f322003bfc0 0x7f322003e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:44.437 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.062+0000 7f322bfff700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f322003bfc0 0x7f322003e3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f321c003a10 tx=0x7f321c0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.062+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 <== mgr.14118 v2:172.21.15.62:6800/766518770 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f322003b2b0 con 0x7f322003bfc0 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 -- 172.21.15.62:0/2318171923 --> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f323410a8f0 con 0x7f322003bfc0 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f3231ffb700 1 -- 172.21.15.62:0/2318171923 <== mgr.14118 v2:172.21.15.62:6800/766518770 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f323410a8f0 con 0x7f322003bfc0 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 -- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f322003bfc0 msgr2=0x7f322003e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f322003bfc0 0x7f322003e3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f321c003a10 tx=0x7f321c0092b0 comp rx=0 tx=0).stop 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 -- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 msgr2=0x7f323419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f323419c6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3224009130 tx=0x7f3224011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 -- 172.21.15.62:0/2318171923 shutdown_connections 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f322003bfc0 0x7f322003e3b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32341071c0 0x7f323419c6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 --2- 172.21.15.62:0/2318171923 >> [v2:172.21.15.62:6800/3409257313,v1:172.21.15.62:6801/3409257313] conn(0x7f32200386e0 0x7f322003aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 -- 172.21.15.62:0/2318171923 >> 172.21.15.62:0/2318171923 conn(0x7f3234100bd0 msgr2=0x7f3234101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 -- 172.21.15.62:0/2318171923 shutdown_connections 2024-05-08T22:29:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.069+0000 7f323a992700 1 -- 172.21.15.62:0/2318171923 wait complete. 2024-05-08T22:29:44.439 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 4 is available 2024-05-08T22:29:44.439 INFO:teuthology.orchestra.run.smithi062.stdout:Setting orchestrator backend to cephadm... 2024-05-08T22:29:45.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.793+0000 7fd6601f3700 1 Processor -- start 2024-05-08T22:29:45.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.793+0000 7fd6601f3700 1 -- start start 2024-05-08T22:29:45.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.794+0000 7fd6601f3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd658109620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:45.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.794+0000 7fd6601f3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6580747a0 con 0x7fd658107230 2024-05-08T22:29:45.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.794+0000 7fd65df8f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd658109620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.794+0000 7fd65df8f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd658109620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59686/0 (socket says 172.21.15.62:59686) 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.794+0000 7fd65df8f700 1 -- 172.21.15.62:0/3340926928 learned_addr learned my addr 172.21.15.62:0/3340926928 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.794+0000 7fd65df8f700 1 -- 172.21.15.62:0/3340926928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd658109b60 con 0x7fd658107230 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.794+0000 7fd65df8f700 1 --2- 172.21.15.62:0/3340926928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd658109620 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd6540098a0 tx=0x7fd654009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5208273be2a7ca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.794+0000 7fd65cf8d700 1 -- 172.21.15.62:0/3340926928 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd654004160 con 0x7fd658107230 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.794+0000 7fd65cf8d700 1 -- 172.21.15.62:0/3340926928 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6540042c0 con 0x7fd658107230 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.794+0000 7fd65cf8d700 1 -- 172.21.15.62:0/3340926928 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd654011a00 con 0x7fd658107230 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.795+0000 7fd6601f3700 1 -- 172.21.15.62:0/3340926928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 msgr2=0x7fd658109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.795+0000 7fd6601f3700 1 --2- 172.21.15.62:0/3340926928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd658109620 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd6540098a0 tx=0x7fd654009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.795+0000 7fd6601f3700 1 -- 172.21.15.62:0/3340926928 shutdown_connections 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.795+0000 7fd6601f3700 1 --2- 172.21.15.62:0/3340926928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd658109620 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.795+0000 7fd6601f3700 1 -- 172.21.15.62:0/3340926928 >> 172.21.15.62:0/3340926928 conn(0x7fd658100ba0 msgr2=0x7fd658103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:45.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.795+0000 7fd6601f3700 1 -- 172.21.15.62:0/3340926928 shutdown_connections 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.795+0000 7fd6601f3700 1 -- 172.21.15.62:0/3340926928 wait complete. 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.796+0000 7fd6601f3700 1 Processor -- start 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.796+0000 7fd6601f3700 1 -- start start 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.796+0000 7fd6601f3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd65819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.796+0000 7fd6601f3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65819a960 con 0x7fd658107230 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.796+0000 7fd65df8f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd65819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.796+0000 7fd65df8f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd65819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59694/0 (socket says 172.21.15.62:59694) 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.796+0000 7fd65df8f700 1 -- 172.21.15.62:0/1277079784 learned_addr learned my addr 172.21.15.62:0/1277079784 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.796+0000 7fd65df8f700 1 -- 172.21.15.62:0/1277079784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd654009550 con 0x7fd658107230 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.796+0000 7fd65df8f700 1 --2- 172.21.15.62:0/1277079784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd65819a420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd654009fb0 tx=0x7fd654011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.797+0000 7fd64effd700 1 -- 172.21.15.62:0/1277079784 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd654003a40 con 0x7fd658107230 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.797+0000 7fd64effd700 1 -- 172.21.15.62:0/1277079784 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd654003ba0 con 0x7fd658107230 2024-05-08T22:29:45.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.797+0000 7fd6601f3700 1 -- 172.21.15.62:0/1277079784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65819ab60 con 0x7fd658107230 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.797+0000 7fd64effd700 1 -- 172.21.15.62:0/1277079784 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd65401a750 con 0x7fd658107230 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.797+0000 7fd6601f3700 1 -- 172.21.15.62:0/1277079784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65819b000 con 0x7fd658107230 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.798+0000 7fd64effd700 1 -- 172.21.15.62:0/1277079784 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fd654024430 con 0x7fd658107230 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.798+0000 7fd64effd700 1 --2- 172.21.15.62:0/1277079784 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fd644038590 0x7fd64403aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.798+0000 7fd6601f3700 1 -- 172.21.15.62:0/1277079784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65810a8b0 con 0x7fd658107230 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.798+0000 7fd64effd700 1 -- 172.21.15.62:0/1277079784 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd65404c5d0 con 0x7fd658107230 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.798+0000 7fd65d78e700 1 --2- 172.21.15.62:0/1277079784 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fd644038590 0x7fd64403aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.798+0000 7fd65d78e700 1 --2- 172.21.15.62:0/1277079784 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fd644038590 0x7fd64403aa50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd648006fd0 tx=0x7fd648006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:44.803+0000 7fd64effd700 1 -- 172.21.15.62:0/1277079784 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd6540189a0 con 0x7fd658107230 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.022+0000 7fd6601f3700 1 -- 172.21.15.62:0/1277079784 --> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fd658101e00 con 0x7fd644038590 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.033+0000 7fd64effd700 1 -- 172.21.15.62:0/1277079784 <== mgr.14118 v2:172.21.15.62:6800/766518770 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd658101e00 con 0x7fd644038590 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.035+0000 7fd6601f3700 1 -- 172.21.15.62:0/1277079784 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fd644038590 msgr2=0x7fd64403aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:45.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.035+0000 7fd6601f3700 1 --2- 172.21.15.62:0/1277079784 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fd644038590 0x7fd64403aa50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd648006fd0 tx=0x7fd648006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:45.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.035+0000 7fd6601f3700 1 -- 172.21.15.62:0/1277079784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 msgr2=0x7fd65819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:45.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.035+0000 7fd6601f3700 1 --2- 172.21.15.62:0/1277079784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd65819a420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd654009fb0 tx=0x7fd654011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:45.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.035+0000 7fd6601f3700 1 -- 172.21.15.62:0/1277079784 shutdown_connections 2024-05-08T22:29:45.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.035+0000 7fd6601f3700 1 --2- 172.21.15.62:0/1277079784 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fd644038590 0x7fd64403aa50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:45.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.035+0000 7fd6601f3700 1 --2- 172.21.15.62:0/1277079784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd658107230 0x7fd65819a420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:45.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.035+0000 7fd6601f3700 1 -- 172.21.15.62:0/1277079784 >> 172.21.15.62:0/1277079784 conn(0x7fd658100ba0 msgr2=0x7fd6581016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:45.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.035+0000 7fd6601f3700 1 -- 172.21.15.62:0/1277079784 shutdown_connections 2024-05-08T22:29:45.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.035+0000 7fd6601f3700 1 -- 172.21.15.62:0/1277079784 wait complete. 2024-05-08T22:29:46.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:45 smithi062 ceph-mon[42571]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T22:29:46.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:45 smithi062 ceph-mon[42571]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T22:29:46.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:45 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:46.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:45 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:46.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:45 smithi062 ceph-mon[42571]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:29:46.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:45 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:46.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:45 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:29:46.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:45 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:29:46.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-08T22:29:46.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.742+0000 7fa1a3e69700 1 Processor -- start 2024-05-08T22:29:46.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.742+0000 7fa1a3e69700 1 -- start start 2024-05-08T22:29:46.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.742+0000 7fa1a3e69700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c1095b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:46.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.742+0000 7fa1a3e69700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c074820 con 0x7fa19c1071c0 2024-05-08T22:29:46.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.743+0000 7fa1a1c05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c1095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:46.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.743+0000 7fa1a1c05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c1095b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59710/0 (socket says 172.21.15.62:59710) 2024-05-08T22:29:46.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.743+0000 7fa1a1c05700 1 -- 172.21.15.62:0/1894037675 learned_addr learned my addr 172.21.15.62:0/1894037675 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:46.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.743+0000 7fa1a1c05700 1 -- 172.21.15.62:0/1894037675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c109af0 con 0x7fa19c1071c0 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.743+0000 7fa1a1c05700 1 --2- 172.21.15.62:0/1894037675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c1095b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa1980098a0 tx=0x7fa198009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f51400ea14599b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.743+0000 7fa1a0c03700 1 -- 172.21.15.62:0/1894037675 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa198004160 con 0x7fa19c1071c0 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.743+0000 7fa1a0c03700 1 -- 172.21.15.62:0/1894037675 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa1980042c0 con 0x7fa19c1071c0 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.743+0000 7fa1a0c03700 1 -- 172.21.15.62:0/1894037675 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa198011a00 con 0x7fa19c1071c0 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.744+0000 7fa1a3e69700 1 -- 172.21.15.62:0/1894037675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 msgr2=0x7fa19c1095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.744+0000 7fa1a3e69700 1 --2- 172.21.15.62:0/1894037675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c1095b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa1980098a0 tx=0x7fa198009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.744+0000 7fa1a3e69700 1 -- 172.21.15.62:0/1894037675 shutdown_connections 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.744+0000 7fa1a3e69700 1 --2- 172.21.15.62:0/1894037675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c1095b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.744+0000 7fa1a3e69700 1 -- 172.21.15.62:0/1894037675 >> 172.21.15.62:0/1894037675 conn(0x7fa19c100bd0 msgr2=0x7fa19c103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.744+0000 7fa1a3e69700 1 -- 172.21.15.62:0/1894037675 shutdown_connections 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.744+0000 7fa1a3e69700 1 -- 172.21.15.62:0/1894037675 wait complete. 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.744+0000 7fa1a3e69700 1 Processor -- start 2024-05-08T22:29:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa1a3e69700 1 -- start start 2024-05-08T22:29:46.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa1a3e69700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:46.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa1a3e69700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c19aa00 con 0x7fa19c1071c0 2024-05-08T22:29:46.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa1a1c05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:46.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa1a1c05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59720/0 (socket says 172.21.15.62:59720) 2024-05-08T22:29:46.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa1a1c05700 1 -- 172.21.15.62:0/3592330941 learned_addr learned my addr 172.21.15.62:0/3592330941 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:46.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa1a1c05700 1 -- 172.21.15.62:0/3592330941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198009550 con 0x7fa19c1071c0 2024-05-08T22:29:46.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa1a1c05700 1 --2- 172.21.15.62:0/3592330941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c19a4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa198009130 tx=0x7fa198011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:46.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa192ffd700 1 -- 172.21.15.62:0/3592330941 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1980038b0 con 0x7fa19c1071c0 2024-05-08T22:29:46.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa192ffd700 1 -- 172.21.15.62:0/3592330941 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa198003a10 con 0x7fa19c1071c0 2024-05-08T22:29:46.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa192ffd700 1 -- 172.21.15.62:0/3592330941 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa198019550 con 0x7fa19c1071c0 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa1a3e69700 1 -- 172.21.15.62:0/3592330941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19c19ac00 con 0x7fa19c1071c0 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.745+0000 7fa1a3e69700 1 -- 172.21.15.62:0/3592330941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19c19b0a0 con 0x7fa19c1071c0 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.746+0000 7fa192ffd700 1 -- 172.21.15.62:0/3592330941 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fa1980197c0 con 0x7fa19c1071c0 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.747+0000 7fa192ffd700 1 --2- 172.21.15.62:0/3592330941 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fa1880385c0 0x7fa18803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.747+0000 7fa1a3e69700 1 -- 172.21.15.62:0/3592330941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19c10a8f0 con 0x7fa19c1071c0 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.747+0000 7fa192ffd700 1 -- 172.21.15.62:0/3592330941 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa19804c5f0 con 0x7fa19c1071c0 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.747+0000 7fa1a1404700 1 --2- 172.21.15.62:0/3592330941 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fa1880385c0 0x7fa18803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.747+0000 7fa1a1404700 1 --2- 172.21.15.62:0/3592330941 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fa1880385c0 0x7fa18803aa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa18c006fd0 tx=0x7fa18c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.752+0000 7fa192ffd700 1 -- 172.21.15.62:0/3592330941 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa198023b40 con 0x7fa19c1071c0 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.957+0000 7fa1a3e69700 1 -- 172.21.15.62:0/3592330941 --> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fa19c101ef0 con 0x7fa1880385c0 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.958+0000 7fa192ffd700 1 -- 172.21.15.62:0/3592330941 <== mgr.14118 v2:172.21.15.62:6800/766518770 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fa19c101ef0 con 0x7fa1880385c0 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.960+0000 7fa1a3e69700 1 -- 172.21.15.62:0/3592330941 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fa1880385c0 msgr2=0x7fa18803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.960+0000 7fa1a3e69700 1 --2- 172.21.15.62:0/3592330941 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fa1880385c0 0x7fa18803aa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa18c006fd0 tx=0x7fa18c006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:46.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.960+0000 7fa1a3e69700 1 -- 172.21.15.62:0/3592330941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 msgr2=0x7fa19c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:46.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.960+0000 7fa1a3e69700 1 --2- 172.21.15.62:0/3592330941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c19a4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa198009130 tx=0x7fa198011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:46.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.960+0000 7fa1a3e69700 1 -- 172.21.15.62:0/3592330941 shutdown_connections 2024-05-08T22:29:46.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.960+0000 7fa1a3e69700 1 --2- 172.21.15.62:0/3592330941 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fa1880385c0 0x7fa18803aa80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:46.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.960+0000 7fa1a3e69700 1 --2- 172.21.15.62:0/3592330941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa19c1071c0 0x7fa19c19a4c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:46.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.960+0000 7fa1a3e69700 1 -- 172.21.15.62:0/3592330941 >> 172.21.15.62:0/3592330941 conn(0x7fa19c100bd0 msgr2=0x7fa19c1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:46.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.960+0000 7fa1a3e69700 1 -- 172.21.15.62:0/3592330941 shutdown_connections 2024-05-08T22:29:46.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:45.961+0000 7fa1a3e69700 1 -- 172.21.15.62:0/3592330941 wait complete. 2024-05-08T22:29:46.317 INFO:teuthology.orchestra.run.smithi062.stdout:Generating ssh key... 2024-05-08T22:29:47.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:47 smithi062 ceph-mon[42571]: [08/May/2024:22:29:45] ENGINE Bus STARTING 2024-05-08T22:29:47.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:47 smithi062 ceph-mon[42571]: [08/May/2024:22:29:45] ENGINE Serving on http://172.21.15.62:8765 2024-05-08T22:29:47.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:47 smithi062 ceph-mon[42571]: [08/May/2024:22:29:45] ENGINE Serving on https://172.21.15.62:7150 2024-05-08T22:29:47.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:47 smithi062 ceph-mon[42571]: [08/May/2024:22:29:45] ENGINE Bus STARTED 2024-05-08T22:29:47.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:47 smithi062 ceph-mon[42571]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:29:47.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:47 smithi062 ceph-mon[42571]: mgrmap e7: x(active, since 2s) 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: Generating public/private rsa key pair. 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: Your identification has been saved in /tmp/tmpt2t_5iiv/key. 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: Your public key has been saved in /tmp/tmpt2t_5iiv/key.pub. 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: The key fingerprint is: 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: SHA256:vyJ2pMICxlPDKSXCWGkjQT+ekQ9DEqvvBYBk19kIknM ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: The key's randomart image is: 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: +---[RSA 3072]----+ 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: |=X+=o + | 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: |B=@E.+ . | 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: |++*O. | 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: |.o.=B | 2024-05-08T22:29:47.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: |o +o.. S | 2024-05-08T22:29:47.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: |.= . .. | 2024-05-08T22:29:47.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: |..o.. o . | 2024-05-08T22:29:47.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: | ...o + o . | 2024-05-08T22:29:47.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: | .. o o .. | 2024-05-08T22:29:47.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: +----[SHA256]-----+ 2024-05-08T22:29:47.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.665+0000 7f2653bf3700 1 Processor -- start 2024-05-08T22:29:47.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.666+0000 7f2653bf3700 1 -- start start 2024-05-08T22:29:47.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.666+0000 7f2653bf3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c106b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:47.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.666+0000 7f2653bf3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c107110 con 0x7f264c106720 2024-05-08T22:29:47.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.666+0000 7f265198f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c106b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:47.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.666+0000 7f265198f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c106b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54412/0 (socket says 172.21.15.62:54412) 2024-05-08T22:29:47.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.666+0000 7f265198f700 1 -- 172.21.15.62:0/2041371803 learned_addr learned my addr 172.21.15.62:0/2041371803 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:47.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.666+0000 7f265198f700 1 -- 172.21.15.62:0/2041371803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c107920 con 0x7f264c106720 2024-05-08T22:29:47.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.667+0000 7f265198f700 1 --2- 172.21.15.62:0/2041371803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c106b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26480098a0 tx=0x7f2648009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472d1736a5eae8dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.667+0000 7f265098d700 1 -- 172.21.15.62:0/2041371803 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2648004160 con 0x7f264c106720 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.667+0000 7f265098d700 1 -- 172.21.15.62:0/2041371803 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f26480042c0 con 0x7f264c106720 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.667+0000 7f265098d700 1 -- 172.21.15.62:0/2041371803 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2648011a00 con 0x7f264c106720 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.667+0000 7f2653bf3700 1 -- 172.21.15.62:0/2041371803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 msgr2=0x7f264c106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.667+0000 7f2653bf3700 1 --2- 172.21.15.62:0/2041371803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c106b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26480098a0 tx=0x7f2648009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.667+0000 7f2653bf3700 1 -- 172.21.15.62:0/2041371803 shutdown_connections 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.667+0000 7f2653bf3700 1 --2- 172.21.15.62:0/2041371803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c106b40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.667+0000 7f2653bf3700 1 -- 172.21.15.62:0/2041371803 >> 172.21.15.62:0/2041371803 conn(0x7f264c101d00 msgr2=0x7f264c104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.667+0000 7f2653bf3700 1 -- 172.21.15.62:0/2041371803 shutdown_connections 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.667+0000 7f2653bf3700 1 -- 172.21.15.62:0/2041371803 wait complete. 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.668+0000 7f2653bf3700 1 Processor -- start 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.668+0000 7f2653bf3700 1 -- start start 2024-05-08T22:29:47.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.668+0000 7f2653bf3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c07bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.668+0000 7f2653bf3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c07c060 con 0x7f264c106720 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.668+0000 7f265198f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c07bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.668+0000 7f265198f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c07bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54418/0 (socket says 172.21.15.62:54418) 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.668+0000 7f265198f700 1 -- 172.21.15.62:0/3562363841 learned_addr learned my addr 172.21.15.62:0/3562363841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.669+0000 7f265198f700 1 -- 172.21.15.62:0/3562363841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2648009550 con 0x7f264c106720 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.669+0000 7f265198f700 1 --2- 172.21.15.62:0/3562363841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c07bb20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2648009130 tx=0x7f2648011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.669+0000 7f263effd700 1 -- 172.21.15.62:0/3562363841 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2648003ab0 con 0x7f264c106720 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.669+0000 7f2653bf3700 1 -- 172.21.15.62:0/3562363841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264c07a2a0 con 0x7f264c106720 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.669+0000 7f263effd700 1 -- 172.21.15.62:0/3562363841 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2648003c10 con 0x7f264c106720 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.669+0000 7f263effd700 1 -- 172.21.15.62:0/3562363841 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26480247d0 con 0x7f264c106720 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.669+0000 7f2653bf3700 1 -- 172.21.15.62:0/3562363841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c07a740 con 0x7f264c106720 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.670+0000 7f263effd700 1 -- 172.21.15.62:0/3562363841 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 7) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f264802d3f0 con 0x7f264c106720 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.670+0000 7f2653bf3700 1 -- 172.21.15.62:0/3562363841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264c0625d0 con 0x7f264c106720 2024-05-08T22:29:47.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.670+0000 7f263effd700 1 --2- 172.21.15.62:0/3562363841 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2638038700 0x7f263803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.670+0000 7f263effd700 1 -- 172.21.15.62:0/3562363841 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f264804c860 con 0x7f264c106720 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.670+0000 7f265118e700 1 --2- 172.21.15.62:0/3562363841 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2638038700 0x7f263803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.670+0000 7f265118e700 1 --2- 172.21.15.62:0/3562363841 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2638038700 0x7f263803abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2640006fd0 tx=0x7f2640006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.675+0000 7f263effd700 1 -- 172.21.15.62:0/3562363841 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2648010740 con 0x7f264c106720 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:46.877+0000 7f2653bf3700 1 -- 172.21.15.62:0/3562363841 --> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f264c07ad80 con 0x7f2638038700 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.194+0000 7f263effd700 1 -- 172.21.15.62:0/3562363841 <== mgr.14118 v2:172.21.15.62:6800/766518770 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f264c07ad80 con 0x7f2638038700 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.196+0000 7f2653bf3700 1 -- 172.21.15.62:0/3562363841 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2638038700 msgr2=0x7f263803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.196+0000 7f2653bf3700 1 --2- 172.21.15.62:0/3562363841 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2638038700 0x7f263803abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2640006fd0 tx=0x7f2640006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.196+0000 7f2653bf3700 1 -- 172.21.15.62:0/3562363841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 msgr2=0x7f264c07bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.196+0000 7f2653bf3700 1 --2- 172.21.15.62:0/3562363841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c07bb20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2648009130 tx=0x7f2648011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.197+0000 7f2653bf3700 1 -- 172.21.15.62:0/3562363841 shutdown_connections 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.197+0000 7f2653bf3700 1 --2- 172.21.15.62:0/3562363841 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2638038700 0x7f263803abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:47.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.197+0000 7f2653bf3700 1 --2- 172.21.15.62:0/3562363841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f264c106720 0x7f264c07bb20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:47.558 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.197+0000 7f2653bf3700 1 -- 172.21.15.62:0/3562363841 >> 172.21.15.62:0/3562363841 conn(0x7f264c101d00 msgr2=0x7f264c1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:47.558 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.197+0000 7f2653bf3700 1 -- 172.21.15.62:0/3562363841 shutdown_connections 2024-05-08T22:29:47.558 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.197+0000 7f2653bf3700 1 -- 172.21.15.62:0/3562363841 wait complete. 2024-05-08T22:29:48.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:48 smithi062 ceph-mon[42571]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:29:48.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:48 smithi062 ceph-mon[42571]: Generating ssh key... 2024-05-08T22:29:48.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:48 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:48.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:48 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:48.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWOjB9Ltp5gYBnuRoFQVO9TKYFzYiYiBE0iP9I/NGPjHm5ap7zGthnvqE+ExhRuQ3+S03RLp/hZ45Wsyu31/EJP+6Q+vuSgWwmala7eRQVOgjM6jnvXKbFFfTw7lcuw6TZQjXE6oVV8dLM5FkF+e9cOj46adtdEgI0QgVqrZ8Gf3YfsIu1PtH7YPScaRt5/jQFz+Lav3Dxr9btSriMHAZ+7AIRkn5qoQh6kFiopsrGFTm0tDn373h8k9t9zwQx8ZkjxZ6WC9MZKJipIWoPnOekJmoDxMKH4fUPmRKBio7rg8F57Oyi+HBP50BFyPlRKjOIehIUWbErJFRfWWFquGE55shmOzOsuTKqDhu6wS9giVyI8Vcg2xWnLecpyGvVQ7t+nVVfehyQ/ATXhIkSnUYGz1aXzHOqTrRRJHdVLOK7UyvpAq6xsdLVnhqw5SdJsCBOUd1JqdopV3WXN8qQCjEWc2/goo9qHZVaiJQchFhdrAja7f9RHjAfumF/ITWvtR0= ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:29:48.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.882+0000 7f2372713700 1 Processor -- start 2024-05-08T22:29:48.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.882+0000 7f2372713700 1 -- start start 2024-05-08T22:29:48.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.883+0000 7f2372713700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c104e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:48.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.883+0000 7f2372713700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c105420 con 0x7f236c104a30 2024-05-08T22:29:48.463 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.883+0000 7f236bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c104e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.883+0000 7f236bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c104e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54426/0 (socket says 172.21.15.62:54426) 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.883+0000 7f236bfff700 1 -- 172.21.15.62:0/1209960403 learned_addr learned my addr 172.21.15.62:0/1209960403 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.883+0000 7f236bfff700 1 -- 172.21.15.62:0/1209960403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c1079c0 con 0x7f236c104a30 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.883+0000 7f236bfff700 1 --2- 172.21.15.62:0/1209960403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c104e50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f235c0098a0 tx=0x7f235c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fed69a8d7ec6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.883+0000 7f236affd700 1 -- 172.21.15.62:0/1209960403 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f235c004160 con 0x7f236c104a30 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.883+0000 7f236affd700 1 -- 172.21.15.62:0/1209960403 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f235c0042c0 con 0x7f236c104a30 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.883+0000 7f236affd700 1 -- 172.21.15.62:0/1209960403 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f235c011a00 con 0x7f236c104a30 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.884+0000 7f2372713700 1 -- 172.21.15.62:0/1209960403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 msgr2=0x7f236c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.884+0000 7f2372713700 1 --2- 172.21.15.62:0/1209960403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c104e50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f235c0098a0 tx=0x7f235c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.884+0000 7f2372713700 1 -- 172.21.15.62:0/1209960403 shutdown_connections 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.884+0000 7f2372713700 1 --2- 172.21.15.62:0/1209960403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c104e50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.884+0000 7f2372713700 1 -- 172.21.15.62:0/1209960403 >> 172.21.15.62:0/1209960403 conn(0x7f236c100340 msgr2=0x7f236c1027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.884+0000 7f2372713700 1 -- 172.21.15.62:0/1209960403 shutdown_connections 2024-05-08T22:29:48.464 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.884+0000 7f2372713700 1 -- 172.21.15.62:0/1209960403 wait complete. 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.885+0000 7f2372713700 1 Processor -- start 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.885+0000 7f2372713700 1 -- start start 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.885+0000 7f2372713700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.885+0000 7f2372713700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c19aa10 con 0x7f236c104a30 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.885+0000 7f236bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.885+0000 7f236bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54442/0 (socket says 172.21.15.62:54442) 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.885+0000 7f236bfff700 1 -- 172.21.15.62:0/1328036786 learned_addr learned my addr 172.21.15.62:0/1328036786 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.886+0000 7f236bfff700 1 -- 172.21.15.62:0/1328036786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c009550 con 0x7f236c104a30 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.886+0000 7f236bfff700 1 --2- 172.21.15.62:0/1328036786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c19a4d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f235c009fb0 tx=0x7f235c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.886+0000 7f23697fa700 1 -- 172.21.15.62:0/1328036786 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f235c003a40 con 0x7f236c104a30 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.886+0000 7f23697fa700 1 -- 172.21.15.62:0/1328036786 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f235c003ba0 con 0x7f236c104a30 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.886+0000 7f23697fa700 1 -- 172.21.15.62:0/1328036786 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f235c01a750 con 0x7f236c104a30 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.886+0000 7f2372713700 1 -- 172.21.15.62:0/1328036786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c19ac10 con 0x7f236c104a30 2024-05-08T22:29:48.465 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.886+0000 7f2372713700 1 -- 172.21.15.62:0/1328036786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c19b0b0 con 0x7f236c104a30 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.887+0000 7f23697fa700 1 -- 172.21.15.62:0/1328036786 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 7) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f235c01a980 con 0x7f236c104a30 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.887+0000 7f23697fa700 1 --2- 172.21.15.62:0/1328036786 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2354038700 0x7f235403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.887+0000 7f23697fa700 1 -- 172.21.15.62:0/1328036786 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f235c04c690 con 0x7f236c104a30 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.887+0000 7f2372713700 1 -- 172.21.15.62:0/1328036786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c10abd0 con 0x7f236c104a30 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.887+0000 7f236b7fe700 1 --2- 172.21.15.62:0/1328036786 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2354038700 0x7f235403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.888+0000 7f236b7fe700 1 --2- 172.21.15.62:0/1328036786 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2354038700 0x7f235403abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2360006fd0 tx=0x7f2360006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:47.892+0000 7f23697fa700 1 -- 172.21.15.62:0/1328036786 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f235c01ac30 con 0x7f236c104a30 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.095+0000 7f2372713700 1 -- 172.21.15.62:0/1328036786 --> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f236c101670 con 0x7f2354038700 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.096+0000 7f23697fa700 1 -- 172.21.15.62:0/1328036786 <== mgr.14118 v2:172.21.15.62:6800/766518770 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f236c101670 con 0x7f2354038700 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.098+0000 7f2372713700 1 -- 172.21.15.62:0/1328036786 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2354038700 msgr2=0x7f235403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.098+0000 7f2372713700 1 --2- 172.21.15.62:0/1328036786 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2354038700 0x7f235403abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2360006fd0 tx=0x7f2360006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.098+0000 7f2372713700 1 -- 172.21.15.62:0/1328036786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 msgr2=0x7f236c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:48.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.098+0000 7f2372713700 1 --2- 172.21.15.62:0/1328036786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c19a4d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f235c009fb0 tx=0x7f235c011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:48.467 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.098+0000 7f2372713700 1 -- 172.21.15.62:0/1328036786 shutdown_connections 2024-05-08T22:29:48.467 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.098+0000 7f2372713700 1 --2- 172.21.15.62:0/1328036786 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2354038700 0x7f235403abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:48.467 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.098+0000 7f2372713700 1 --2- 172.21.15.62:0/1328036786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c104a30 0x7f236c19a4d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:48.467 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.098+0000 7f2372713700 1 -- 172.21.15.62:0/1328036786 >> 172.21.15.62:0/1328036786 conn(0x7f236c100340 msgr2=0x7f236c100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:48.467 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.099+0000 7f2372713700 1 -- 172.21.15.62:0/1328036786 shutdown_connections 2024-05-08T22:29:48.467 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.099+0000 7f2372713700 1 -- 172.21.15.62:0/1328036786 wait complete. 2024-05-08T22:29:48.467 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-08T22:29:48.467 INFO:teuthology.orchestra.run.smithi062.stdout:Adding key to root@localhost authorized_keys... 2024-05-08T22:29:48.467 INFO:teuthology.orchestra.run.smithi062.stdout:Adding host smithi062... 2024-05-08T22:29:49.349 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:49 smithi062 ceph-mon[42571]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:29:50.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:50 smithi062 ceph-mon[42571]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:29:50.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:50 smithi062 ceph-mon[42571]: Deploying cephadm binary to smithi062 2024-05-08T22:29:51.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Added host 'smithi062' with addr '172.21.15.62' 2024-05-08T22:29:51.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.853+0000 7f6d81830700 1 Processor -- start 2024-05-08T22:29:51.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.853+0000 7f6d81830700 1 -- start start 2024-05-08T22:29:51.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.853+0000 7f6d81830700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c109570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:51.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.853+0000 7f6d81830700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c079eb0 con 0x7f6d7c107180 2024-05-08T22:29:51.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.853+0000 7f6d7affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:51.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.853+0000 7f6d7affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54454/0 (socket says 172.21.15.62:54454) 2024-05-08T22:29:51.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.853+0000 7f6d7affd700 1 -- 172.21.15.62:0/3328389700 learned_addr learned my addr 172.21.15.62:0/3328389700 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:51.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.854+0000 7f6d7affd700 1 -- 172.21.15.62:0/3328389700 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c109ab0 con 0x7f6d7c107180 2024-05-08T22:29:51.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.854+0000 7f6d7affd700 1 --2- 172.21.15.62:0/3328389700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c109570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0098a0 tx=0x7f6d6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffebafb895104f20 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:51.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.854+0000 7f6d79ffb700 1 -- 172.21.15.62:0/3328389700 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d6c004160 con 0x7f6d7c107180 2024-05-08T22:29:51.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.854+0000 7f6d79ffb700 1 -- 172.21.15.62:0/3328389700 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d6c0042c0 con 0x7f6d7c107180 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.854+0000 7f6d79ffb700 1 -- 172.21.15.62:0/3328389700 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d6c011a00 con 0x7f6d7c107180 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.854+0000 7f6d81830700 1 -- 172.21.15.62:0/3328389700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 msgr2=0x7f6d7c109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.854+0000 7f6d81830700 1 --2- 172.21.15.62:0/3328389700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c109570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0098a0 tx=0x7f6d6c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.854+0000 7f6d81830700 1 -- 172.21.15.62:0/3328389700 shutdown_connections 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.854+0000 7f6d81830700 1 --2- 172.21.15.62:0/3328389700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c109570 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.854+0000 7f6d81830700 1 -- 172.21.15.62:0/3328389700 >> 172.21.15.62:0/3328389700 conn(0x7f6d7c100bb0 msgr2=0x7f6d7c102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.855+0000 7f6d81830700 1 -- 172.21.15.62:0/3328389700 shutdown_connections 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.855+0000 7f6d81830700 1 -- 172.21.15.62:0/3328389700 wait complete. 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.855+0000 7f6d81830700 1 Processor -- start 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.855+0000 7f6d81830700 1 -- start start 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.855+0000 7f6d81830700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c1069b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.855+0000 7f6d81830700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c106ef0 con 0x7f6d7c107180 2024-05-08T22:29:51.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.856+0000 7f6d7affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c1069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:51.625 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.856+0000 7f6d7affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c1069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54460/0 (socket says 172.21.15.62:54460) 2024-05-08T22:29:51.625 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.856+0000 7f6d7affd700 1 -- 172.21.15.62:0/698039894 learned_addr learned my addr 172.21.15.62:0/698039894 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:51.625 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.856+0000 7f6d7affd700 1 -- 172.21.15.62:0/698039894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c009550 con 0x7f6d7c107180 2024-05-08T22:29:51.625 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.856+0000 7f6d7affd700 1 --2- 172.21.15.62:0/698039894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c1069b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009130 tx=0x7f6d6c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:51.625 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.856+0000 7f6d8082e700 1 -- 172.21.15.62:0/698039894 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d6c003b70 con 0x7f6d7c107180 2024-05-08T22:29:51.625 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.856+0000 7f6d8082e700 1 -- 172.21.15.62:0/698039894 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d6c003cd0 con 0x7f6d7c107180 2024-05-08T22:29:51.625 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.856+0000 7f6d81830700 1 -- 172.21.15.62:0/698039894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7c105130 con 0x7f6d7c107180 2024-05-08T22:29:51.625 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.856+0000 7f6d8082e700 1 -- 172.21.15.62:0/698039894 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d6c0197d0 con 0x7f6d7c107180 2024-05-08T22:29:51.625 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.856+0000 7f6d81830700 1 -- 172.21.15.62:0/698039894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7c1055d0 con 0x7f6d7c107180 2024-05-08T22:29:51.625 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.857+0000 7f6d8082e700 1 -- 172.21.15.62:0/698039894 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 7) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f6d6c0199c0 con 0x7f6d7c107180 2024-05-08T22:29:51.625 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.857+0000 7f6d8082e700 1 --2- 172.21.15.62:0/698039894 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d64038750 0x7f6d6403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.857+0000 7f6d8082e700 1 -- 172.21.15.62:0/698039894 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6d6c04cb10 con 0x7f6d7c107180 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.857+0000 7f6d7a7fc700 1 --2- 172.21.15.62:0/698039894 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d64038750 0x7f6d6403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.858+0000 7f6d81830700 1 -- 172.21.15.62:0/698039894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7c0625d0 con 0x7f6d7c107180 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.858+0000 7f6d7a7fc700 1 --2- 172.21.15.62:0/698039894 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d64038750 0x7f6d6403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6d70006fd0 tx=0x7f6d70006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:48.863+0000 7f6d8082e700 1 -- 172.21.15.62:0/698039894 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6d6c018db0 con 0x7f6d7c107180 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:49.068+0000 7f6d81830700 1 -- 172.21.15.62:0/698039894 --> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d7c105ac0 con 0x7f6d64038750 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:51.310+0000 7f6d8082e700 1 -- 172.21.15.62:0/698039894 <== mgr.14118 v2:172.21.15.62:6800/766518770 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f6d7c105ac0 con 0x7f6d64038750 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:51.312+0000 7f6d81830700 1 -- 172.21.15.62:0/698039894 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d64038750 msgr2=0x7f6d6403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:51.312+0000 7f6d81830700 1 --2- 172.21.15.62:0/698039894 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d64038750 0x7f6d6403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6d70006fd0 tx=0x7f6d70006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:51.313+0000 7f6d81830700 1 -- 172.21.15.62:0/698039894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 msgr2=0x7f6d7c1069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:51.313+0000 7f6d81830700 1 --2- 172.21.15.62:0/698039894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c1069b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009130 tx=0x7f6d6c011ad0 comp rx=0 tx=0).stop 2024-05-08T22:29:51.626 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:51.313+0000 7f6d81830700 1 -- 172.21.15.62:0/698039894 shutdown_connections 2024-05-08T22:29:51.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:51.313+0000 7f6d81830700 1 --2- 172.21.15.62:0/698039894 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d64038750 0x7f6d6403ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:51.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:51.313+0000 7f6d81830700 1 --2- 172.21.15.62:0/698039894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c107180 0x7f6d7c1069b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:51.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:51.313+0000 7f6d81830700 1 -- 172.21.15.62:0/698039894 >> 172.21.15.62:0/698039894 conn(0x7f6d7c100bb0 msgr2=0x7f6d7c1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:51.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:51.313+0000 7f6d81830700 1 -- 172.21.15.62:0/698039894 shutdown_connections 2024-05-08T22:29:51.627 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:51.313+0000 7f6d81830700 1 -- 172.21.15.62:0/698039894 wait complete. 2024-05-08T22:29:51.627 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mon service... 2024-05-08T22:29:52.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:52 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:52.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:52 smithi062 ceph-mon[42571]: Added host smithi062 2024-05-08T22:29:52.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:52 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:29:52.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:52 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:52.774 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-08T22:29:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.023+0000 7efd68a71700 1 Processor -- start 2024-05-08T22:29:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.023+0000 7efd68a71700 1 -- start start 2024-05-08T22:29:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.024+0000 7efd68a71700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd64079210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.024+0000 7efd68a71700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd640797e0 con 0x7efd6407adb0 2024-05-08T22:29:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.024+0000 7efd6259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd64079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.024+0000 7efd6259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd64079210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54474/0 (socket says 172.21.15.62:54474) 2024-05-08T22:29:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.024+0000 7efd6259c700 1 -- 172.21.15.62:0/4108242070 learned_addr learned my addr 172.21.15.62:0/4108242070 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.024+0000 7efd6259c700 1 -- 172.21.15.62:0/4108242070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd64079920 con 0x7efd6407adb0 2024-05-08T22:29:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.024+0000 7efd6259c700 1 --2- 172.21.15.62:0/4108242070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd64079210 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efd54009e80 tx=0x7efd5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb3649dec58b349 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.024+0000 7efd6159a700 1 -- 172.21.15.62:0/4108242070 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd5400e070 con 0x7efd6407adb0 2024-05-08T22:29:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.024+0000 7efd6159a700 1 -- 172.21.15.62:0/4108242070 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd5400b8c0 con 0x7efd6407adb0 2024-05-08T22:29:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.024+0000 7efd6159a700 1 -- 172.21.15.62:0/4108242070 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd540044c0 con 0x7efd6407adb0 2024-05-08T22:29:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.025+0000 7efd68a71700 1 -- 172.21.15.62:0/4108242070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 msgr2=0x7efd64079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.025+0000 7efd68a71700 1 --2- 172.21.15.62:0/4108242070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd64079210 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efd54009e80 tx=0x7efd5400b330 comp rx=0 tx=0).stop 2024-05-08T22:29:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.025+0000 7efd68a71700 1 -- 172.21.15.62:0/4108242070 shutdown_connections 2024-05-08T22:29:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.025+0000 7efd68a71700 1 --2- 172.21.15.62:0/4108242070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd64079210 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.025+0000 7efd68a71700 1 -- 172.21.15.62:0/4108242070 >> 172.21.15.62:0/4108242070 conn(0x7efd64101d00 msgr2=0x7efd64104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.025+0000 7efd68a71700 1 -- 172.21.15.62:0/4108242070 shutdown_connections 2024-05-08T22:29:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.025+0000 7efd68a71700 1 -- 172.21.15.62:0/4108242070 wait complete. 2024-05-08T22:29:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.026+0000 7efd68a71700 1 Processor -- start 2024-05-08T22:29:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.026+0000 7efd68a71700 1 -- start start 2024-05-08T22:29:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.026+0000 7efd68a71700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd6419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.026+0000 7efd68a71700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6419cb40 con 0x7efd6407adb0 2024-05-08T22:29:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.026+0000 7efd6259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd6419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.026+0000 7efd6259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd6419c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54490/0 (socket says 172.21.15.62:54490) 2024-05-08T22:29:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.026+0000 7efd6259c700 1 -- 172.21.15.62:0/4011188276 learned_addr learned my addr 172.21.15.62:0/4011188276 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.027+0000 7efd6259c700 1 -- 172.21.15.62:0/4011188276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54009b30 con 0x7efd6407adb0 2024-05-08T22:29:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.027+0000 7efd6259c700 1 --2- 172.21.15.62:0/4011188276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd6419c600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efd54009fd0 tx=0x7efd540045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.027+0000 7efd5b7fe700 1 -- 172.21.15.62:0/4011188276 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd5400e040 con 0x7efd6407adb0 2024-05-08T22:29:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.027+0000 7efd5b7fe700 1 -- 172.21.15.62:0/4011188276 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd5400bdf0 con 0x7efd6407adb0 2024-05-08T22:29:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.027+0000 7efd68a71700 1 -- 172.21.15.62:0/4011188276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6419cd40 con 0x7efd6407adb0 2024-05-08T22:29:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.027+0000 7efd5b7fe700 1 -- 172.21.15.62:0/4011188276 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd5401c7a0 con 0x7efd6407adb0 2024-05-08T22:29:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.027+0000 7efd68a71700 1 -- 172.21.15.62:0/4011188276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6419d240 con 0x7efd6407adb0 2024-05-08T22:29:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.028+0000 7efd68a71700 1 -- 172.21.15.62:0/4011188276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6410cad0 con 0x7efd6407adb0 2024-05-08T22:29:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.028+0000 7efd5b7fe700 1 -- 172.21.15.62:0/4011188276 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 7) v1 ==== 45541+0+0 (secure 0 0 0) 0x7efd54012b20 con 0x7efd6407adb0 2024-05-08T22:29:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.028+0000 7efd5b7fe700 1 --2- 172.21.15.62:0/4011188276 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7efd50038750 0x7efd5003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.029+0000 7efd5b7fe700 1 -- 172.21.15.62:0/4011188276 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efd5404c6e0 con 0x7efd6407adb0 2024-05-08T22:29:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.029+0000 7efd61d9b700 1 --2- 172.21.15.62:0/4011188276 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7efd50038750 0x7efd5003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.029+0000 7efd61d9b700 1 --2- 172.21.15.62:0/4011188276 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7efd50038750 0x7efd5003ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efd4c006fd0 tx=0x7efd4c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.033+0000 7efd5b7fe700 1 -- 172.21.15.62:0/4011188276 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efd5401cd10 con 0x7efd6407adb0 2024-05-08T22:29:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.251+0000 7efd68a71700 1 -- 172.21.15.62:0/4011188276 --> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7efd64102f20 con 0x7efd50038750 2024-05-08T22:29:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.256+0000 7efd5b7fe700 1 -- 172.21.15.62:0/4011188276 <== mgr.14118 v2:172.21.15.62:6800/766518770 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efd64102f20 con 0x7efd50038750 2024-05-08T22:29:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.258+0000 7efd68a71700 1 -- 172.21.15.62:0/4011188276 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7efd50038750 msgr2=0x7efd5003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.258+0000 7efd68a71700 1 --2- 172.21.15.62:0/4011188276 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7efd50038750 0x7efd5003ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efd4c006fd0 tx=0x7efd4c006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.258+0000 7efd68a71700 1 -- 172.21.15.62:0/4011188276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 msgr2=0x7efd6419c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.258+0000 7efd68a71700 1 --2- 172.21.15.62:0/4011188276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd6419c600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efd54009fd0 tx=0x7efd540045f0 comp rx=0 tx=0).stop 2024-05-08T22:29:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.258+0000 7efd68a71700 1 -- 172.21.15.62:0/4011188276 shutdown_connections 2024-05-08T22:29:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.258+0000 7efd68a71700 1 --2- 172.21.15.62:0/4011188276 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7efd50038750 0x7efd5003ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.258+0000 7efd68a71700 1 --2- 172.21.15.62:0/4011188276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd6407adb0 0x7efd6419c600 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.258+0000 7efd68a71700 1 -- 172.21.15.62:0/4011188276 >> 172.21.15.62:0/4011188276 conn(0x7efd64101d00 msgr2=0x7efd64102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.258+0000 7efd68a71700 1 -- 172.21.15.62:0/4011188276 shutdown_connections 2024-05-08T22:29:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:52.258+0000 7efd68a71700 1 -- 172.21.15.62:0/4011188276 wait complete. 2024-05-08T22:29:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mgr service... 2024-05-08T22:29:53.938 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-08T22:29:53.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.170+0000 7f45253cb700 1 Processor -- start 2024-05-08T22:29:53.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.170+0000 7f45253cb700 1 -- start start 2024-05-08T22:29:53.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.170+0000 7f45253cb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f4520105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:53.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.170+0000 7f45253cb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4520074780 con 0x7f4520105040 2024-05-08T22:29:53.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.170+0000 7f451effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f4520105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:53.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.170+0000 7f451effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f4520105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54492/0 (socket says 172.21.15.62:54492) 2024-05-08T22:29:53.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.170+0000 7f451effd700 1 -- 172.21.15.62:0/4126850101 learned_addr learned my addr 172.21.15.62:0/4126850101 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:53.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.170+0000 7f451effd700 1 -- 172.21.15.62:0/4126850101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45200748c0 con 0x7f4520105040 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.171+0000 7f451effd700 1 --2- 172.21.15.62:0/4126850101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f4520105460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f45100098a0 tx=0x7f4510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc25c9e2f955880 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.171+0000 7f451dffb700 1 -- 172.21.15.62:0/4126850101 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4510004160 con 0x7f4520105040 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.171+0000 7f451dffb700 1 -- 172.21.15.62:0/4126850101 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45100042c0 con 0x7f4520105040 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.171+0000 7f451dffb700 1 -- 172.21.15.62:0/4126850101 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45100119b0 con 0x7f4520105040 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.171+0000 7f45253cb700 1 -- 172.21.15.62:0/4126850101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 msgr2=0x7f4520105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.171+0000 7f45253cb700 1 --2- 172.21.15.62:0/4126850101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f4520105460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f45100098a0 tx=0x7f4510009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.171+0000 7f45253cb700 1 -- 172.21.15.62:0/4126850101 shutdown_connections 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.171+0000 7f45253cb700 1 --2- 172.21.15.62:0/4126850101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f4520105460 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.171+0000 7f45253cb700 1 -- 172.21.15.62:0/4126850101 >> 172.21.15.62:0/4126850101 conn(0x7f4520100ba0 msgr2=0x7f4520103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.171+0000 7f45253cb700 1 -- 172.21.15.62:0/4126850101 shutdown_connections 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.172+0000 7f45253cb700 1 -- 172.21.15.62:0/4126850101 wait complete. 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.172+0000 7f45253cb700 1 Processor -- start 2024-05-08T22:29:53.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.172+0000 7f45253cb700 1 -- start start 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.172+0000 7f45253cb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f452019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.172+0000 7f45253cb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452019cbb0 con 0x7f4520105040 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.173+0000 7f451effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f452019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.173+0000 7f451effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f452019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54498/0 (socket says 172.21.15.62:54498) 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.173+0000 7f451effd700 1 -- 172.21.15.62:0/3662532871 learned_addr learned my addr 172.21.15.62:0/3662532871 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.173+0000 7f451effd700 1 -- 172.21.15.62:0/3662532871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4510009550 con 0x7f4520105040 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.173+0000 7f451effd700 1 --2- 172.21.15.62:0/3662532871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f452019c670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4510006af0 tx=0x7f4510011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.173+0000 7f4507fff700 1 -- 172.21.15.62:0/3662532871 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4510003a40 con 0x7f4520105040 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.173+0000 7f4507fff700 1 -- 172.21.15.62:0/3662532871 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4510003ba0 con 0x7f4520105040 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.173+0000 7f45253cb700 1 -- 172.21.15.62:0/3662532871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452019cdb0 con 0x7f4520105040 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.173+0000 7f4507fff700 1 -- 172.21.15.62:0/3662532871 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45100196c0 con 0x7f4520105040 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.173+0000 7f45253cb700 1 -- 172.21.15.62:0/3662532871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452019d250 con 0x7f4520105040 2024-05-08T22:29:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.174+0000 7f4507fff700 1 -- 172.21.15.62:0/3662532871 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 7) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f4510019980 con 0x7f4520105040 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.174+0000 7f4507fff700 1 --2- 172.21.15.62:0/3662532871 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f45080386b0 0x7f450803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.174+0000 7f45253cb700 1 -- 172.21.15.62:0/3662532871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45201963c0 con 0x7f4520105040 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.174+0000 7f4507fff700 1 -- 172.21.15.62:0/3662532871 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f451004d5c0 con 0x7f4520105040 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.174+0000 7f451e7fc700 1 --2- 172.21.15.62:0/3662532871 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f45080386b0 0x7f450803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.175+0000 7f451e7fc700 1 --2- 172.21.15.62:0/3662532871 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f45080386b0 0x7f450803ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4514006fd0 tx=0x7f4514006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.179+0000 7f4507fff700 1 -- 172.21.15.62:0/3662532871 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f45100243f0 con 0x7f4520105040 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.419+0000 7f45253cb700 1 -- 172.21.15.62:0/3662532871 --> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f452002cf50 con 0x7f45080386b0 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.428+0000 7f4507fff700 1 -- 172.21.15.62:0/3662532871 <== mgr.14118 v2:172.21.15.62:6800/766518770 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f452002cf50 con 0x7f45080386b0 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.430+0000 7f45253cb700 1 -- 172.21.15.62:0/3662532871 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f45080386b0 msgr2=0x7f450803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.430+0000 7f45253cb700 1 --2- 172.21.15.62:0/3662532871 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f45080386b0 0x7f450803ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4514006fd0 tx=0x7f4514006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.430+0000 7f45253cb700 1 -- 172.21.15.62:0/3662532871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 msgr2=0x7f452019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.430+0000 7f45253cb700 1 --2- 172.21.15.62:0/3662532871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f452019c670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4510006af0 tx=0x7f4510011a10 comp rx=0 tx=0).stop 2024-05-08T22:29:53.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.430+0000 7f45253cb700 1 -- 172.21.15.62:0/3662532871 shutdown_connections 2024-05-08T22:29:53.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.430+0000 7f45253cb700 1 --2- 172.21.15.62:0/3662532871 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f45080386b0 0x7f450803ab70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:53.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.430+0000 7f45253cb700 1 --2- 172.21.15.62:0/3662532871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4520105040 0x7f452019c670 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:53.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.430+0000 7f45253cb700 1 -- 172.21.15.62:0/3662532871 >> 172.21.15.62:0/3662532871 conn(0x7f4520100ba0 msgr2=0x7f4520193500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:53.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.430+0000 7f45253cb700 1 -- 172.21.15.62:0/3662532871 shutdown_connections 2024-05-08T22:29:53.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:53.430+0000 7f45253cb700 1 -- 172.21.15.62:0/3662532871 wait complete. 2024-05-08T22:29:54.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:53 smithi062 ceph-mon[42571]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:29:54.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:53 smithi062 ceph-mon[42571]: Saving service mon spec with placement count:5 2024-05-08T22:29:54.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:53 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:54.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:53 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:54.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:53 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:54.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:54 smithi062 ceph-mon[42571]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:29:54.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:54 smithi062 ceph-mon[42571]: Saving service mgr spec with placement count:2 2024-05-08T22:29:54.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:54 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:54.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:54 smithi062 ceph-mon[42571]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:29:54.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:54 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3229075238' entity='client.admin' 2024-05-08T22:29:54.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.342+0000 7fece330d700 1 Processor -- start 2024-05-08T22:29:54.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.343+0000 7fece330d700 1 -- start start 2024-05-08T22:29:54.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.343+0000 7fece330d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc1051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:54.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.343+0000 7fece330d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc1057b0 con 0x7fecdc104dc0 2024-05-08T22:29:54.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.343+0000 7fece10a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc1051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:54.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.343+0000 7fece10a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc1051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54500/0 (socket says 172.21.15.62:54500) 2024-05-08T22:29:54.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.343+0000 7fece10a9700 1 -- 172.21.15.62:0/1732770838 learned_addr learned my addr 172.21.15.62:0/1732770838 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:54.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.343+0000 7fece10a9700 1 -- 172.21.15.62:0/1732770838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc105fc0 con 0x7fecdc104dc0 2024-05-08T22:29:54.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.344+0000 7fece10a9700 1 --2- 172.21.15.62:0/1732770838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc1051e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fecd800aa40 tx=0x7fecd800ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797fcf6686152c00 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:54.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.344+0000 7fecd3fff700 1 -- 172.21.15.62:0/1732770838 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecd8004160 con 0x7fecdc104dc0 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.344+0000 7fecd3fff700 1 -- 172.21.15.62:0/1732770838 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecd800cdb0 con 0x7fecdc104dc0 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.344+0000 7fecd3fff700 1 -- 172.21.15.62:0/1732770838 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecd8003c30 con 0x7fecdc104dc0 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.344+0000 7fece330d700 1 -- 172.21.15.62:0/1732770838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 msgr2=0x7fecdc1051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.344+0000 7fece330d700 1 --2- 172.21.15.62:0/1732770838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc1051e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fecd800aa40 tx=0x7fecd800ad50 comp rx=0 tx=0).stop 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.345+0000 7fece330d700 1 -- 172.21.15.62:0/1732770838 shutdown_connections 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.345+0000 7fece330d700 1 --2- 172.21.15.62:0/1732770838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc1051e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.345+0000 7fece330d700 1 -- 172.21.15.62:0/1732770838 >> 172.21.15.62:0/1732770838 conn(0x7fecdc100340 msgr2=0x7fecdc1027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.345+0000 7fece330d700 1 -- 172.21.15.62:0/1732770838 shutdown_connections 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.345+0000 7fece330d700 1 -- 172.21.15.62:0/1732770838 wait complete. 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.345+0000 7fece330d700 1 Processor -- start 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.345+0000 7fece330d700 1 -- start start 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fece330d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fece330d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc19aa40 con 0x7fecdc104dc0 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fece10a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fece10a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54508/0 (socket says 172.21.15.62:54508) 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fece10a9700 1 -- 172.21.15.62:0/3229075238 learned_addr learned my addr 172.21.15.62:0/3229075238 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fece10a9700 1 -- 172.21.15.62:0/3229075238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd800a6f0 con 0x7fecdc104dc0 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fece10a9700 1 --2- 172.21.15.62:0/3229075238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc19a500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fecd800afd0 tx=0x7fecd80041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fecd27fc700 1 -- 172.21.15.62:0/3229075238 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecd8010520 con 0x7fecdc104dc0 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fecd27fc700 1 -- 172.21.15.62:0/3229075238 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecd8010b40 con 0x7fecdc104dc0 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fece330d700 1 -- 172.21.15.62:0/3229075238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc19ac40 con 0x7fecdc104dc0 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fecd27fc700 1 -- 172.21.15.62:0/3229075238 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecd8021a80 con 0x7fecdc104dc0 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.346+0000 7fece330d700 1 -- 172.21.15.62:0/3229075238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc19b0e0 con 0x7fecdc104dc0 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.348+0000 7fecd27fc700 1 -- 172.21.15.62:0/3229075238 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 7) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fecd802b440 con 0x7fecdc104dc0 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.348+0000 7fece330d700 1 -- 172.21.15.62:0/3229075238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecdc194150 con 0x7fecdc104dc0 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.348+0000 7fecd27fc700 1 --2- 172.21.15.62:0/3229075238 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fecc8038700 0x7fecc803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:54.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.348+0000 7fecd27fc700 1 -- 172.21.15.62:0/3229075238 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fecd804cb20 con 0x7fecdc104dc0 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.348+0000 7fece08a8700 1 --2- 172.21.15.62:0/3229075238 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fecc8038700 0x7fecc803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.348+0000 7fece08a8700 1 --2- 172.21.15.62:0/3229075238 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fecc8038700 0x7fecc803abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feccc006fd0 tx=0x7feccc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.353+0000 7fecd27fc700 1 -- 172.21.15.62:0/3229075238 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fecd8029770 con 0x7fecdc104dc0 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.557+0000 7fece330d700 1 -- 172.21.15.62:0/3229075238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fecdc02cef0 con 0x7fecdc104dc0 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.563+0000 7fecd27fc700 1 -- 172.21.15.62:0/3229075238 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fecd8026070 con 0x7fecdc104dc0 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.565+0000 7fece330d700 1 -- 172.21.15.62:0/3229075238 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fecc8038700 msgr2=0x7fecc803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.565+0000 7fece330d700 1 --2- 172.21.15.62:0/3229075238 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fecc8038700 0x7fecc803abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feccc006fd0 tx=0x7feccc006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.565+0000 7fece330d700 1 -- 172.21.15.62:0/3229075238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 msgr2=0x7fecdc19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.565+0000 7fece330d700 1 --2- 172.21.15.62:0/3229075238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc19a500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fecd800afd0 tx=0x7fecd80041d0 comp rx=0 tx=0).stop 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.565+0000 7fece330d700 1 -- 172.21.15.62:0/3229075238 shutdown_connections 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.565+0000 7fece330d700 1 --2- 172.21.15.62:0/3229075238 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7fecc8038700 0x7fecc803abc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.565+0000 7fece330d700 1 --2- 172.21.15.62:0/3229075238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc104dc0 0x7fecdc19a500 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.565+0000 7fece330d700 1 -- 172.21.15.62:0/3229075238 >> 172.21.15.62:0/3229075238 conn(0x7fecdc100340 msgr2=0x7fecdc191290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.566+0000 7fece330d700 1 -- 172.21.15.62:0/3229075238 shutdown_connections 2024-05-08T22:29:54.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:54.566+0000 7fece330d700 1 -- 172.21.15.62:0/3229075238 wait complete. 2024-05-08T22:29:56.225 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.644+0000 7f3e0e50c700 1 Processor -- start 2024-05-08T22:29:56.900 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.644+0000 7f3e0e50c700 1 -- start start 2024-05-08T22:29:56.901 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.645+0000 7f3e0e50c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e081053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:56.901 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.645+0000 7f3e0e50c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e08079e50 con 0x7f3e08104fd0 2024-05-08T22:29:56.901 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.645+0000 7f3e07fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e081053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:56.901 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.645+0000 7f3e07fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e081053f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54522/0 (socket says 172.21.15.62:54522) 2024-05-08T22:29:56.901 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.645+0000 7f3e07fff700 1 -- 172.21.15.62:0/1385989445 learned_addr learned my addr 172.21.15.62:0/1385989445 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:56.901 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.645+0000 7f3e07fff700 1 -- 172.21.15.62:0/1385989445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e08105930 con 0x7f3e08104fd0 2024-05-08T22:29:56.901 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.646+0000 7f3e07fff700 1 --2- 172.21.15.62:0/1385989445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e081053f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3df80098a0 tx=0x7f3df8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fd21f2198e10ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:56.902 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.646+0000 7f3e06ffd700 1 -- 172.21.15.62:0/1385989445 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3df8004160 con 0x7f3e08104fd0 2024-05-08T22:29:56.902 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.646+0000 7f3e06ffd700 1 -- 172.21.15.62:0/1385989445 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3df80042c0 con 0x7f3e08104fd0 2024-05-08T22:29:56.902 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.646+0000 7f3e06ffd700 1 -- 172.21.15.62:0/1385989445 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3df80119b0 con 0x7f3e08104fd0 2024-05-08T22:29:56.902 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.646+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1385989445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 msgr2=0x7f3e081053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:56.902 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.646+0000 7f3e0e50c700 1 --2- 172.21.15.62:0/1385989445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e081053f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3df80098a0 tx=0x7f3df8009bb0 comp rx=0 tx=0).stop 2024-05-08T22:29:56.902 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.646+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1385989445 shutdown_connections 2024-05-08T22:29:56.903 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.646+0000 7f3e0e50c700 1 --2- 172.21.15.62:0/1385989445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e081053f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:56.903 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.646+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1385989445 >> 172.21.15.62:0/1385989445 conn(0x7f3e08100bf0 msgr2=0x7f3e08103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:56.903 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.646+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1385989445 shutdown_connections 2024-05-08T22:29:56.903 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.647+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1385989445 wait complete. 2024-05-08T22:29:56.903 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.647+0000 7f3e0e50c700 1 Processor -- start 2024-05-08T22:29:56.903 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.647+0000 7f3e0e50c700 1 -- start start 2024-05-08T22:29:56.903 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.647+0000 7f3e0e50c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e0819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:56.903 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.647+0000 7f3e0e50c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0819cba0 con 0x7f3e08104fd0 2024-05-08T22:29:56.903 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.648+0000 7f3e07fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e0819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:56.903 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.648+0000 7f3e07fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e0819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54538/0 (socket says 172.21.15.62:54538) 2024-05-08T22:29:56.904 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.648+0000 7f3e07fff700 1 -- 172.21.15.62:0/1367795997 learned_addr learned my addr 172.21.15.62:0/1367795997 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:56.904 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.648+0000 7f3e07fff700 1 -- 172.21.15.62:0/1367795997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df8009550 con 0x7f3e08104fd0 2024-05-08T22:29:56.904 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.648+0000 7f3e07fff700 1 --2- 172.21.15.62:0/1367795997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e0819c660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006af0 tx=0x7f3df8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:56.904 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.648+0000 7f3e057fa700 1 -- 172.21.15.62:0/1367795997 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3df8003a40 con 0x7f3e08104fd0 2024-05-08T22:29:56.904 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.648+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1367795997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0819cda0 con 0x7f3e08104fd0 2024-05-08T22:29:56.904 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.648+0000 7f3e057fa700 1 -- 172.21.15.62:0/1367795997 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3df8003ba0 con 0x7f3e08104fd0 2024-05-08T22:29:56.904 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.648+0000 7f3e057fa700 1 -- 172.21.15.62:0/1367795997 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3df80196c0 con 0x7f3e08104fd0 2024-05-08T22:29:56.904 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.648+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1367795997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0819d240 con 0x7f3e08104fd0 2024-05-08T22:29:56.904 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.649+0000 7f3e057fa700 1 -- 172.21.15.62:0/1367795997 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 7) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f3df8019980 con 0x7f3e08104fd0 2024-05-08T22:29:56.905 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.649+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1367795997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e081963f0 con 0x7f3e08104fd0 2024-05-08T22:29:56.905 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.649+0000 7f3e057fa700 1 --2- 172.21.15.62:0/1367795997 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f3df00386b0 0x7f3df003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:56.905 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.649+0000 7f3e057fa700 1 -- 172.21.15.62:0/1367795997 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3df804c870 con 0x7f3e08104fd0 2024-05-08T22:29:56.905 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.650+0000 7f3e077fe700 1 --2- 172.21.15.62:0/1367795997 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f3df00386b0 0x7f3df003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:56.905 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.650+0000 7f3e077fe700 1 --2- 172.21.15.62:0/1367795997 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f3df00386b0 0x7f3df003ab70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc006fd0 tx=0x7f3dfc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:56.905 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.654+0000 7f3e057fa700 1 -- 172.21.15.62:0/1367795997 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3df8018760 con 0x7f3e08104fd0 2024-05-08T22:29:56.905 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.855+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1367795997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f3e081069b0 con 0x7f3e08104fd0 2024-05-08T22:29:56.905 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.858+0000 7f3e057fa700 1 -- 172.21.15.62:0/1367795997 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f3df8023390 con 0x7f3e08104fd0 2024-05-08T22:29:56.905 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.860+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1367795997 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f3df00386b0 msgr2=0x7f3df003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:56.906 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.860+0000 7f3e0e50c700 1 --2- 172.21.15.62:0/1367795997 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f3df00386b0 0x7f3df003ab70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc006fd0 tx=0x7f3dfc006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:56.906 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.860+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1367795997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 msgr2=0x7f3e0819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:56.906 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.860+0000 7f3e0e50c700 1 --2- 172.21.15.62:0/1367795997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e0819c660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006af0 tx=0x7f3df8011a10 comp rx=0 tx=0).stop 2024-05-08T22:29:56.906 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.860+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1367795997 shutdown_connections 2024-05-08T22:29:56.906 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.860+0000 7f3e0e50c700 1 --2- 172.21.15.62:0/1367795997 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f3df00386b0 0x7f3df003ab70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:56.906 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.860+0000 7f3e0e50c700 1 --2- 172.21.15.62:0/1367795997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08104fd0 0x7f3e0819c660 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:56.906 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.860+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1367795997 >> 172.21.15.62:0/1367795997 conn(0x7f3e08100bf0 msgr2=0x7f3e08193530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:56.906 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.860+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1367795997 shutdown_connections 2024-05-08T22:29:56.907 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:55.860+0000 7f3e0e50c700 1 -- 172.21.15.62:0/1367795997 wait complete. 2024-05-08T22:29:56.907 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling the dashboard module... 2024-05-08T22:29:57.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:56 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1367795997' entity='client.admin' 2024-05-08T22:29:57.161 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:56 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1045871488' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T22:29:57.162 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:56 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: ignoring --setuser ceph since I am not root 2024-05-08T22:29:57.162 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:56 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: ignoring --setgroup ceph since I am not root 2024-05-08T22:29:57.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.567+0000 7f2f9df22700 1 Processor -- start 2024-05-08T22:29:57.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.567+0000 7f2f9df22700 1 -- start start 2024-05-08T22:29:57.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.568+0000 7f2f9df22700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f98109680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:57.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.568+0000 7f2f9df22700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f98074740 con 0x7f2f98107290 2024-05-08T22:29:57.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.568+0000 7f2f977fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f98109680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:57.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.568+0000 7f2f977fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f98109680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34632/0 (socket says 172.21.15.62:34632) 2024-05-08T22:29:57.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.568+0000 7f2f977fe700 1 -- 172.21.15.62:0/874102700 learned_addr learned my addr 172.21.15.62:0/874102700 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:57.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.568+0000 7f2f977fe700 1 -- 172.21.15.62:0/874102700 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f98074880 con 0x7f2f98107290 2024-05-08T22:29:57.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.568+0000 7f2f977fe700 1 --2- 172.21.15.62:0/874102700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f98109680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009e80 tx=0x7f2f8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28fca833234da9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:57.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.568+0000 7f2f967fc700 1 -- 172.21.15.62:0/874102700 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f8800e070 con 0x7f2f98107290 2024-05-08T22:29:57.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.568+0000 7f2f967fc700 1 -- 172.21.15.62:0/874102700 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f8800b8c0 con 0x7f2f98107290 2024-05-08T22:29:57.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.568+0000 7f2f967fc700 1 -- 172.21.15.62:0/874102700 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f880043e0 con 0x7f2f98107290 2024-05-08T22:29:57.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.569+0000 7f2f9df22700 1 -- 172.21.15.62:0/874102700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 msgr2=0x7f2f98109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:57.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.569+0000 7f2f9df22700 1 --2- 172.21.15.62:0/874102700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f98109680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009e80 tx=0x7f2f8800b330 comp rx=0 tx=0).stop 2024-05-08T22:29:57.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.569+0000 7f2f9df22700 1 -- 172.21.15.62:0/874102700 shutdown_connections 2024-05-08T22:29:57.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.569+0000 7f2f9df22700 1 --2- 172.21.15.62:0/874102700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f98109680 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:57.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.569+0000 7f2f9df22700 1 -- 172.21.15.62:0/874102700 >> 172.21.15.62:0/874102700 conn(0x7f2f98100ba0 msgr2=0x7f2f98103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:57.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.569+0000 7f2f9df22700 1 -- 172.21.15.62:0/874102700 shutdown_connections 2024-05-08T22:29:57.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.569+0000 7f2f9df22700 1 -- 172.21.15.62:0/874102700 wait complete. 2024-05-08T22:29:57.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.570+0000 7f2f9df22700 1 Processor -- start 2024-05-08T22:29:57.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.570+0000 7f2f9df22700 1 -- start start 2024-05-08T22:29:57.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.570+0000 7f2f9df22700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f9819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:57.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.570+0000 7f2f9df22700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9819cb30 con 0x7f2f98107290 2024-05-08T22:29:57.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.570+0000 7f2f977fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f9819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:57.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.570+0000 7f2f977fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f9819c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34644/0 (socket says 172.21.15.62:34644) 2024-05-08T22:29:57.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.570+0000 7f2f977fe700 1 -- 172.21.15.62:0/1045871488 learned_addr learned my addr 172.21.15.62:0/1045871488 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:57.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.570+0000 7f2f977fe700 1 -- 172.21.15.62:0/1045871488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f88009b30 con 0x7f2f98107290 2024-05-08T22:29:57.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.570+0000 7f2f977fe700 1 --2- 172.21.15.62:0/1045871488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f9819c5f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009e50 tx=0x7f2f88003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:57.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.571+0000 7f2f94ff9700 1 -- 172.21.15.62:0/1045871488 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f8800e040 con 0x7f2f98107290 2024-05-08T22:29:57.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.571+0000 7f2f9df22700 1 -- 172.21.15.62:0/1045871488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9819cd30 con 0x7f2f98107290 2024-05-08T22:29:57.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.571+0000 7f2f94ff9700 1 -- 172.21.15.62:0/1045871488 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f8800bd70 con 0x7f2f98107290 2024-05-08T22:29:57.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.571+0000 7f2f94ff9700 1 -- 172.21.15.62:0/1045871488 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f8801c6b0 con 0x7f2f98107290 2024-05-08T22:29:57.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.571+0000 7f2f9df22700 1 -- 172.21.15.62:0/1045871488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9819d230 con 0x7f2f98107290 2024-05-08T22:29:57.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.572+0000 7f2f9df22700 1 -- 172.21.15.62:0/1045871488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9810a930 con 0x7f2f98107290 2024-05-08T22:29:57.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.572+0000 7f2f94ff9700 1 -- 172.21.15.62:0/1045871488 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 7) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f2f88012430 con 0x7f2f98107290 2024-05-08T22:29:57.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.572+0000 7f2f94ff9700 1 --2- 172.21.15.62:0/1045871488 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2f840386b0 0x7f2f8403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:57.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.572+0000 7f2f94ff9700 1 -- 172.21.15.62:0/1045871488 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2f8804c6d0 con 0x7f2f98107290 2024-05-08T22:29:57.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.572+0000 7f2f96ffd700 1 --2- 172.21.15.62:0/1045871488 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2f840386b0 0x7f2f8403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:57.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.573+0000 7f2f96ffd700 1 --2- 172.21.15.62:0/1045871488 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2f840386b0 0x7f2f8403ab70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2f80006fd0 tx=0x7f2f80006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:57.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.577+0000 7f2f94ff9700 1 -- 172.21.15.62:0/1045871488 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2f88011390 con 0x7f2f98107290 2024-05-08T22:29:57.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.837+0000 7f2f9df22700 1 -- 172.21.15.62:0/1045871488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f2f9810ab70 con 0x7f2f98107290 2024-05-08T22:29:57.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.865+0000 7f2f94ff9700 1 -- 172.21.15.62:0/1045871488 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 8) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f2f88012ad0 con 0x7f2f98107290 2024-05-08T22:29:57.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.866+0000 7f2f94ff9700 1 -- 172.21.15.62:0/1045871488 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v8) v1 ==== 88+0+0 (secure 0 0 0) 0x7f2f880126e0 con 0x7f2f98107290 2024-05-08T22:29:57.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.868+0000 7f2f9df22700 1 -- 172.21.15.62:0/1045871488 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2f840386b0 msgr2=0x7f2f8403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:57.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.868+0000 7f2f9df22700 1 --2- 172.21.15.62:0/1045871488 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2f840386b0 0x7f2f8403ab70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2f80006fd0 tx=0x7f2f80006e40 comp rx=0 tx=0).stop 2024-05-08T22:29:57.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.868+0000 7f2f9df22700 1 -- 172.21.15.62:0/1045871488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 msgr2=0x7f2f9819c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:57.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.868+0000 7f2f9df22700 1 --2- 172.21.15.62:0/1045871488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f9819c5f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009e50 tx=0x7f2f88003d00 comp rx=0 tx=0).stop 2024-05-08T22:29:57.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.869+0000 7f2f9df22700 1 -- 172.21.15.62:0/1045871488 shutdown_connections 2024-05-08T22:29:57.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.869+0000 7f2f9df22700 1 --2- 172.21.15.62:0/1045871488 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f2f840386b0 0x7f2f8403ab70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:57.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.869+0000 7f2f9df22700 1 --2- 172.21.15.62:0/1045871488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f98107290 0x7f2f9819c5f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:57.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.869+0000 7f2f9df22700 1 -- 172.21.15.62:0/1045871488 >> 172.21.15.62:0/1045871488 conn(0x7f2f98100ba0 msgr2=0x7f2f98101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:57.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.869+0000 7f2f9df22700 1 -- 172.21.15.62:0/1045871488 shutdown_connections 2024-05-08T22:29:57.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:56.869+0000 7f2f9df22700 1 -- 172.21.15.62:0/1045871488 wait complete. 2024-05-08T22:29:57.414 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:57 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:57.332+0000 7f6808f7d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:29:57.865 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:57 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:57.435+0000 7f6808f7d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:29:57.865 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:57 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:57.528+0000 7f6808f7d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:29:57.865 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:57 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:57.619+0000 7f6808f7d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:29:58.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:57 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1045871488' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T22:29:58.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:57 smithi062 ceph-mon[42571]: mgrmap e8: x(active, since 13s) 2024-05-08T22:29:58.246 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 8, 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.610+0000 7f26c15eb700 1 Processor -- start 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.610+0000 7f26c15eb700 1 -- start start 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.610+0000 7f26c15eb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc105a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.611+0000 7f26c15eb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc106040 con 0x7f26bc105650 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.611+0000 7f26baffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc105a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.611+0000 7f26baffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc105a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34672/0 (socket says 172.21.15.62:34672) 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.611+0000 7f26baffd700 1 -- 172.21.15.62:0/2596413540 learned_addr learned my addr 172.21.15.62:0/2596413540 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.611+0000 7f26baffd700 1 -- 172.21.15.62:0/2596413540 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc106850 con 0x7f26bc105650 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.611+0000 7f26baffd700 1 --2- 172.21.15.62:0/2596413540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc105a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00bb40 tx=0x7f26ac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc2ead86e880e9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.611+0000 7f26b9ffb700 1 -- 172.21.15.62:0/2596413540 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26ac00dbe0 con 0x7f26bc105650 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.611+0000 7f26b9ffb700 1 -- 172.21.15.62:0/2596413540 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26ac0044f0 con 0x7f26bc105650 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.611+0000 7f26b9ffb700 1 -- 172.21.15.62:0/2596413540 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26ac003e60 con 0x7f26bc105650 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.612+0000 7f26c15eb700 1 -- 172.21.15.62:0/2596413540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 msgr2=0x7f26bc105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.612+0000 7f26c15eb700 1 --2- 172.21.15.62:0/2596413540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc105a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00bb40 tx=0x7f26ac00be50 comp rx=0 tx=0).stop 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.612+0000 7f26c15eb700 1 -- 172.21.15.62:0/2596413540 shutdown_connections 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.612+0000 7f26c15eb700 1 --2- 172.21.15.62:0/2596413540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc105a70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.612+0000 7f26c15eb700 1 -- 172.21.15.62:0/2596413540 >> 172.21.15.62:0/2596413540 conn(0x7f26bc100bd0 msgr2=0x7f26bc103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.612+0000 7f26c15eb700 1 -- 172.21.15.62:0/2596413540 shutdown_connections 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.612+0000 7f26c15eb700 1 -- 172.21.15.62:0/2596413540 wait complete. 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.613+0000 7f26c15eb700 1 Processor -- start 2024-05-08T22:29:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.613+0000 7f26c15eb700 1 -- start start 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.613+0000 7f26c15eb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.613+0000 7f26c15eb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc198850 con 0x7f26bc105650 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.613+0000 7f26baffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.613+0000 7f26baffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34686/0 (socket says 172.21.15.62:34686) 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.614+0000 7f26baffd700 1 -- 172.21.15.62:0/4052717337 learned_addr learned my addr 172.21.15.62:0/4052717337 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.614+0000 7f26baffd700 1 -- 172.21.15.62:0/4052717337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac00b7f0 con 0x7f26bc105650 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.614+0000 7f26baffd700 1 --2- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc198310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00b340 tx=0x7f26ac003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.614+0000 7f26a3fff700 1 -- 172.21.15.62:0/4052717337 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26ac01e070 con 0x7f26bc105650 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.614+0000 7f26a3fff700 1 -- 172.21.15.62:0/4052717337 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26ac010b40 con 0x7f26bc105650 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.614+0000 7f26c15eb700 1 -- 172.21.15.62:0/4052717337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26bc198a50 con 0x7f26bc105650 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.614+0000 7f26a3fff700 1 -- 172.21.15.62:0/4052717337 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26ac022ac0 con 0x7f26bc105650 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.614+0000 7f26c15eb700 1 -- 172.21.15.62:0/4052717337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26bc198ef0 con 0x7f26bc105650 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.615+0000 7f26a3fff700 1 -- 172.21.15.62:0/4052717337 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f26ac02c440 con 0x7f26bc105650 2024-05-08T22:29:58.249 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.615+0000 7f26a3fff700 1 --2- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f26a4038750 0x7f26a403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.615+0000 7f26c15eb700 1 -- 172.21.15.62:0/4052717337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26bc073940 con 0x7f26bc105650 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.615+0000 7f26ba7fc700 1 -- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f26a4038750 msgr2=0x7f26a403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/766518770 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.615+0000 7f26a3fff700 1 -- 172.21.15.62:0/4052717337 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f26ac04f0e0 con 0x7f26bc105650 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.615+0000 7f26ba7fc700 1 --2- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f26a4038750 0x7f26a403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.620+0000 7f26a3fff700 1 -- 172.21.15.62:0/4052717337 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f26ac02a720 con 0x7f26bc105650 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.816+0000 7f26ba7fc700 1 -- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f26a4038750 msgr2=0x7f26a403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/766518770 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.816+0000 7f26ba7fc700 1 --2- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f26a4038750 0x7f26a403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.881+0000 7f26c15eb700 1 -- 172.21.15.62:0/4052717337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f26bc191a50 con 0x7f26bc105650 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.881+0000 7f26a3fff700 1 -- 172.21.15.62:0/4052717337 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v8) v1 ==== 56+0+88 (secure 0 0 0) 0x7f26ac027070 con 0x7f26bc105650 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.884+0000 7f26c15eb700 1 -- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f26a4038750 msgr2=0x7f26a403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.884+0000 7f26c15eb700 1 --2- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f26a4038750 0x7f26a403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.884+0000 7f26c15eb700 1 -- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 msgr2=0x7f26bc198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:29:58.250 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.884+0000 7f26c15eb700 1 --2- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc198310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00b340 tx=0x7f26ac003fb0 comp rx=0 tx=0).stop 2024-05-08T22:29:58.251 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.884+0000 7f26c15eb700 1 -- 172.21.15.62:0/4052717337 shutdown_connections 2024-05-08T22:29:58.251 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.884+0000 7f26c15eb700 1 --2- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f26a4038750 0x7f26a403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:58.251 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.884+0000 7f26c15eb700 1 --2- 172.21.15.62:0/4052717337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc105650 0x7f26bc198310 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:29:58.251 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.884+0000 7f26c15eb700 1 -- 172.21.15.62:0/4052717337 >> 172.21.15.62:0/4052717337 conn(0x7f26bc100bd0 msgr2=0x7f26bc18eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:29:58.251 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.884+0000 7f26c15eb700 1 -- 172.21.15.62:0/4052717337 shutdown_connections 2024-05-08T22:29:58.251 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:57.884+0000 7f26c15eb700 1 -- 172.21.15.62:0/4052717337 wait complete. 2024-05-08T22:29:58.251 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2024-05-08T22:29:58.251 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 8... 2024-05-08T22:29:58.568 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:58 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:58.297+0000 7f6808f7d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:29:58.865 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:58 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:58.568+0000 7f6808f7d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:29:58.865 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:58 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:58.659+0000 7f6808f7d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:29:58.865 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:58 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:58.774+0000 7f6808f7d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:29:59.161 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:29:58 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/4052717337' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T22:29:59.161 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:58 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:58.938+0000 7f6808f7d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:29:59.161 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:59 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:59.071+0000 7f6808f7d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:29:59.161 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:59 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:59.160+0000 7f6808f7d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:29:59.921 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:59 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:59.549+0000 7f6808f7d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:30:00.262 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:29:59 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:29:59.959+0000 7f6808f7d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:30:00.262 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:00 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:00.164+0000 7f6808f7d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:30:00.262 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:00 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:00.262+0000 7f6808f7d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:30:01.321 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:01 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:01.064+0000 7f6808f7d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:30:01.321 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:01 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:01.165+0000 7f6808f7d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:30:01.321 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:01 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:01.320+0000 7f6808f7d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:30:02.371 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:01 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:01.978+0000 7f6808f7d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:30:02.371 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:02 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:02.168+0000 7f6808f7d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:30:02.371 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:02 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:02.370+0000 7f6808f7d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:30:03.643 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:03 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:03.343+0000 7f6808f7d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:30:03.921 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:03 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:03.644+0000 7f6808f7d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:30:03.921 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:03 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:03.737+0000 7f6808f7d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:30:04.421 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:03 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:03.938+0000 7f6808f7d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:30:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:04 smithi062 ceph-mon[42571]: Active manager daemon x restarted 2024-05-08T22:30:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:04 smithi062 ceph-mon[42571]: Activating manager daemon x 2024-05-08T22:30:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:04 smithi062 ceph-mon[42571]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T22:30:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:04 smithi062 ceph-mon[42571]: mgrmap e9: x(active, starting, since 0.0034978s) 2024-05-08T22:30:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:30:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:30:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:30:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:30:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:30:04.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:04 smithi062 ceph-mon[42571]: Manager daemon x is now available 2024-05-08T22:30:04.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:05.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:30:05.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:30:05.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:05 smithi062 ceph-mon[42571]: [08/May/2024:22:30:04] ENGINE Bus STARTING 2024-05-08T22:30:05.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:05.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:05 smithi062 ceph-mon[42571]: [08/May/2024:22:30:04] ENGINE Serving on http://172.21.15.62:8765 2024-05-08T22:30:05.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:05 smithi062 ceph-mon[42571]: mgrmap e10: x(active, since 1.00634s) 2024-05-08T22:30:05.312 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-05-08T22:30:05.312 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 10, 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.592+0000 7f6d2198b700 1 Processor -- start 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.593+0000 7f6d2198b700 1 -- start start 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.593+0000 7f6d2198b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c1095b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.593+0000 7f6d2198b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c074820 con 0x7f6d1c1071c0 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.593+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c1095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.593+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c1095b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34700/0 (socket says 172.21.15.62:34700) 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.593+0000 7f6d1affd700 1 -- 172.21.15.62:0/1077047873 learned_addr learned my addr 172.21.15.62:0/1077047873 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.593+0000 7f6d1affd700 1 -- 172.21.15.62:0/1077047873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c109af0 con 0x7f6d1c1071c0 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.593+0000 7f6d1affd700 1 --2- 172.21.15.62:0/1077047873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c1095b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c0098a0 tx=0x7f6d0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a726fed748361f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:05.313 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.594+0000 7f6d19ffb700 1 -- 172.21.15.62:0/1077047873 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d0c004160 con 0x7f6d1c1071c0 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.594+0000 7f6d19ffb700 1 -- 172.21.15.62:0/1077047873 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d0c0042c0 con 0x7f6d1c1071c0 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.594+0000 7f6d19ffb700 1 -- 172.21.15.62:0/1077047873 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d0c0119b0 con 0x7f6d1c1071c0 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.594+0000 7f6d2198b700 1 -- 172.21.15.62:0/1077047873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 msgr2=0x7f6d1c1095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.594+0000 7f6d2198b700 1 --2- 172.21.15.62:0/1077047873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c1095b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c0098a0 tx=0x7f6d0c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.594+0000 7f6d2198b700 1 -- 172.21.15.62:0/1077047873 shutdown_connections 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.594+0000 7f6d2198b700 1 --2- 172.21.15.62:0/1077047873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c1095b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.594+0000 7f6d2198b700 1 -- 172.21.15.62:0/1077047873 >> 172.21.15.62:0/1077047873 conn(0x7f6d1c100bd0 msgr2=0x7f6d1c103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.594+0000 7f6d2198b700 1 -- 172.21.15.62:0/1077047873 shutdown_connections 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.595+0000 7f6d2198b700 1 -- 172.21.15.62:0/1077047873 wait complete. 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.595+0000 7f6d2198b700 1 Processor -- start 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.595+0000 7f6d2198b700 1 -- start start 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.595+0000 7f6d2198b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:05.314 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.595+0000 7f6d2198b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c19aa20 con 0x7f6d1c1071c0 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.596+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.596+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34714/0 (socket says 172.21.15.62:34714) 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.596+0000 7f6d1affd700 1 -- 172.21.15.62:0/3569675016 learned_addr learned my addr 172.21.15.62:0/3569675016 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.596+0000 7f6d1affd700 1 -- 172.21.15.62:0/3569675016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c009550 con 0x7f6d1c1071c0 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.596+0000 7f6d1affd700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c19a4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c003e60 tx=0x7f6d0c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.596+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d0c0038b0 con 0x7f6d1c1071c0 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.596+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d0c003a10 con 0x7f6d1c1071c0 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.596+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d0c019530 con 0x7f6d1c1071c0 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.596+0000 7f6d2198b700 1 -- 172.21.15.62:0/3569675016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c19ac20 con 0x7f6d1c1071c0 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.596+0000 7f6d2198b700 1 -- 172.21.15.62:0/3569675016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c19b0c0 con 0x7f6d1c1071c0 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.597+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f6d0c0197f0 con 0x7f6d1c1071c0 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.597+0000 7f6d20989700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 0x7f6d0403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.597+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 --> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d0403b2d0 con 0x7f6d04038700 2024-05-08T22:30:05.315 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.597+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6d0c04c8a0 con 0x7f6d1c1071c0 2024-05-08T22:30:05.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.597+0000 7f6d1a7fc700 1 -- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 msgr2=0x7f6d0403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/766518770 2024-05-08T22:30:05.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.597+0000 7f6d1a7fc700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 0x7f6d0403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:30:05.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.798+0000 7f6d1a7fc700 1 -- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 msgr2=0x7f6d0403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/766518770 2024-05-08T22:30:05.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:58.798+0000 7f6d1a7fc700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 0x7f6d0403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T22:30:05.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:59.199+0000 7f6d1a7fc700 1 -- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 msgr2=0x7f6d0403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/766518770 2024-05-08T22:30:05.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:29:59.199+0000 7f6d1a7fc700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 0x7f6d0403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T22:30:05.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:00.000+0000 7f6d1a7fc700 1 -- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 msgr2=0x7f6d0403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/766518770 2024-05-08T22:30:05.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:00.000+0000 7f6d1a7fc700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 0x7f6d0403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T22:30:05.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:01.601+0000 7f6d1a7fc700 1 -- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 msgr2=0x7f6d0403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/766518770 2024-05-08T22:30:05.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:01.601+0000 7f6d1a7fc700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 0x7f6d0403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:03.953+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 9) v1 ==== 45321+0+0 (secure 0 0 0) 0x7f6d0c019d20 con 0x7f6d1c1071c0 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:03.953+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 msgr2=0x7f6d0403abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:03.953+0000 7f6d20989700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 0x7f6d0403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.955+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 10) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f6d0c04e320 con 0x7f6d1c1071c0 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.955+0000 7f6d20989700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d0403bff0 0x7f6d0403e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.955+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d0403b2d0 con 0x7f6d0403bff0 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.955+0000 7f6d1a7fc700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d0403bff0 0x7f6d0403e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.956+0000 7f6d1a7fc700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d0403bff0 0x7f6d0403e3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6d10003a10 tx=0x7f6d100092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.957+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f6d0403b2d0 con 0x7f6d0403bff0 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.963+0000 7f6d2198b700 1 -- 172.21.15.62:0/3569675016 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f6d1c10a930 con 0x7f6d0403bff0 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.963+0000 7f6d20989700 1 -- 172.21.15.62:0/3569675016 <== mgr.14150 v2:172.21.15.62:6800/4071146880 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f6d1c10a930 con 0x7f6d0403bff0 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.964+0000 7f6d2198b700 1 -- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d0403bff0 msgr2=0x7f6d0403e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:05.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.964+0000 7f6d2198b700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d0403bff0 0x7f6d0403e3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6d10003a10 tx=0x7f6d100092b0 comp rx=0 tx=0).stop 2024-05-08T22:30:05.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.964+0000 7f6d2198b700 1 -- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 msgr2=0x7f6d1c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:05.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.964+0000 7f6d2198b700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c19a4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c003e60 tx=0x7f6d0c011a30 comp rx=0 tx=0).stop 2024-05-08T22:30:05.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.964+0000 7f6d2198b700 1 -- 172.21.15.62:0/3569675016 shutdown_connections 2024-05-08T22:30:05.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.964+0000 7f6d2198b700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d0403bff0 0x7f6d0403e3e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:05.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.964+0000 7f6d2198b700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c1071c0 0x7f6d1c19a4e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:05.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.964+0000 7f6d2198b700 1 --2- 172.21.15.62:0/3569675016 >> [v2:172.21.15.62:6800/766518770,v1:172.21.15.62:6801/766518770] conn(0x7f6d04038700 0x7f6d0403abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:05.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.964+0000 7f6d2198b700 1 -- 172.21.15.62:0/3569675016 >> 172.21.15.62:0/3569675016 conn(0x7f6d1c100bd0 msgr2=0x7f6d1c1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:05.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.964+0000 7f6d2198b700 1 -- 172.21.15.62:0/3569675016 shutdown_connections 2024-05-08T22:30:05.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:04.964+0000 7f6d2198b700 1 -- 172.21.15.62:0/3569675016 wait complete. 2024-05-08T22:30:05.318 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 8 is available 2024-05-08T22:30:05.318 INFO:teuthology.orchestra.run.smithi062.stdout:Generating a dashboard self-signed certificate... 2024-05-08T22:30:06.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:06 smithi062 ceph-mon[42571]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T22:30:06.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:06 smithi062 ceph-mon[42571]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T22:30:06.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:06 smithi062 ceph-mon[42571]: [08/May/2024:22:30:05] ENGINE Serving on https://172.21.15.62:7150 2024-05-08T22:30:06.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:06 smithi062 ceph-mon[42571]: [08/May/2024:22:30:05] ENGINE Bus STARTED 2024-05-08T22:30:06.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:06.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:06.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:07.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-08T22:30:07.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.694+0000 7fd80e8f0700 1 Processor -- start 2024-05-08T22:30:07.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.694+0000 7fd80e8f0700 1 -- start start 2024-05-08T22:30:07.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.694+0000 7fd80e8f0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd8081095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:07.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.694+0000 7fd80e8f0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080747e0 con 0x7fd808107200 2024-05-08T22:30:07.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.695+0000 7fd807fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd8081095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.695+0000 7fd807fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd8081095f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34762/0 (socket says 172.21.15.62:34762) 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.695+0000 7fd807fff700 1 -- 172.21.15.62:0/3075780402 learned_addr learned my addr 172.21.15.62:0/3075780402 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.695+0000 7fd807fff700 1 -- 172.21.15.62:0/3075780402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd808109b30 con 0x7fd808107200 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.695+0000 7fd807fff700 1 --2- 172.21.15.62:0/3075780402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd8081095f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80098a0 tx=0x7fd7f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6cc05afddfe2c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.695+0000 7fd806ffd700 1 -- 172.21.15.62:0/3075780402 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7f8004160 con 0x7fd808107200 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.695+0000 7fd806ffd700 1 -- 172.21.15.62:0/3075780402 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7f80042c0 con 0x7fd808107200 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.695+0000 7fd806ffd700 1 -- 172.21.15.62:0/3075780402 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7f80119b0 con 0x7fd808107200 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.696+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3075780402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 msgr2=0x7fd8081095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.696+0000 7fd80e8f0700 1 --2- 172.21.15.62:0/3075780402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd8081095f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80098a0 tx=0x7fd7f8009bb0 comp rx=0 tx=0).stop 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.696+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3075780402 shutdown_connections 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.696+0000 7fd80e8f0700 1 --2- 172.21.15.62:0/3075780402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd8081095f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.696+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3075780402 >> 172.21.15.62:0/3075780402 conn(0x7fd808100bd0 msgr2=0x7fd808103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.696+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3075780402 shutdown_connections 2024-05-08T22:30:07.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.697+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3075780402 wait complete. 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.697+0000 7fd80e8f0700 1 Processor -- start 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.697+0000 7fd80e8f0700 1 -- start start 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.698+0000 7fd80e8f0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd80819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.698+0000 7fd80e8f0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80819aa30 con 0x7fd808107200 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.698+0000 7fd807fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd80819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.698+0000 7fd807fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd80819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34772/0 (socket says 172.21.15.62:34772) 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.698+0000 7fd807fff700 1 -- 172.21.15.62:0/3360154069 learned_addr learned my addr 172.21.15.62:0/3360154069 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.698+0000 7fd807fff700 1 -- 172.21.15.62:0/3360154069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f8009550 con 0x7fd808107200 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.699+0000 7fd807fff700 1 --2- 172.21.15.62:0/3360154069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd80819a4f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8003e60 tx=0x7fd7f8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.699+0000 7fd8057fa700 1 -- 172.21.15.62:0/3360154069 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7f80038b0 con 0x7fd808107200 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.699+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3360154069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80819ac30 con 0x7fd808107200 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.699+0000 7fd8057fa700 1 -- 172.21.15.62:0/3360154069 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7f8003a10 con 0x7fd808107200 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.699+0000 7fd8057fa700 1 -- 172.21.15.62:0/3360154069 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7f8019530 con 0x7fd808107200 2024-05-08T22:30:07.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.699+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3360154069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80819b0d0 con 0x7fd808107200 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.700+0000 7fd8057fa700 1 -- 172.21.15.62:0/3360154069 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 10) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fd7f80197f0 con 0x7fd808107200 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.700+0000 7fd8057fa700 1 --2- 172.21.15.62:0/3360154069 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd7f00385e0 0x7fd7f003aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.700+0000 7fd8057fa700 1 -- 172.21.15.62:0/3360154069 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd7f804d7b0 con 0x7fd808107200 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.700+0000 7fd8077fe700 1 --2- 172.21.15.62:0/3360154069 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd7f00385e0 0x7fd7f003aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.700+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3360154069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f4005320 con 0x7fd808107200 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.701+0000 7fd8077fe700 1 --2- 172.21.15.62:0/3360154069 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd7f00385e0 0x7fd7f003aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc006fd0 tx=0x7fd7fc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.705+0000 7fd8057fa700 1 -- 172.21.15.62:0/3360154069 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd7f8019b40 con 0x7fd808107200 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.927+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3360154069 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7f4000bf0 con 0x7fd7f00385e0 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.997+0000 7fd8057fa700 1 -- 172.21.15.62:0/3360154069 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fd7f4000bf0 con 0x7fd7f00385e0 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.999+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3360154069 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd7f00385e0 msgr2=0x7fd7f003aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.999+0000 7fd80e8f0700 1 --2- 172.21.15.62:0/3360154069 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd7f00385e0 0x7fd7f003aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc006fd0 tx=0x7fd7fc006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.999+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3360154069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 msgr2=0x7fd80819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:07.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.999+0000 7fd80e8f0700 1 --2- 172.21.15.62:0/3360154069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd80819a4f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8003e60 tx=0x7fd7f8011a30 comp rx=0 tx=0).stop 2024-05-08T22:30:07.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.999+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3360154069 shutdown_connections 2024-05-08T22:30:07.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.999+0000 7fd80e8f0700 1 --2- 172.21.15.62:0/3360154069 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd7f00385e0 0x7fd7f003aaa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:07.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.999+0000 7fd80e8f0700 1 --2- 172.21.15.62:0/3360154069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808107200 0x7fd80819a4f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:07.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:05.999+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3360154069 >> 172.21.15.62:0/3360154069 conn(0x7fd808100bd0 msgr2=0x7fd8081017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:07.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:06.000+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3360154069 shutdown_connections 2024-05-08T22:30:07.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:06.000+0000 7fd80e8f0700 1 -- 172.21.15.62:0/3360154069 wait complete. 2024-05-08T22:30:07.606 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial admin user... 2024-05-08T22:30:07.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:07 smithi062 ceph-mon[42571]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:07.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:07 smithi062 ceph-mon[42571]: mgrmap e11: x(active, since 2s) 2024-05-08T22:30:08.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$BKNvqXyw8TzTXpS97y9BQe3Y3l3DB8ShoI1kApnBDvwtZrOh8JDmW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715207408, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-08T22:30:08.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.947+0000 7f0518963700 1 Processor -- start 2024-05-08T22:30:08.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.947+0000 7f0518963700 1 -- start start 2024-05-08T22:30:08.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.947+0000 7f0518963700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f0514105f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:08.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.947+0000 7f0518963700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0514106530 con 0x7f0514105b40 2024-05-08T22:30:08.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.947+0000 7f051259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f0514105f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:08.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.947+0000 7f051259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f0514105f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50622/0 (socket says 172.21.15.62:50622) 2024-05-08T22:30:08.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.947+0000 7f051259c700 1 -- 172.21.15.62:0/355053045 learned_addr learned my addr 172.21.15.62:0/355053045 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:08.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.948+0000 7f051259c700 1 -- 172.21.15.62:0/355053045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0514106860 con 0x7f0514105b40 2024-05-08T22:30:08.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.948+0000 7f051259c700 1 --2- 172.21.15.62:0/355053045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f0514105f60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05040098a0 tx=0x7f0504009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a8ee2920b52f43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.948+0000 7f051159a700 1 -- 172.21.15.62:0/355053045 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0504004160 con 0x7f0514105b40 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.948+0000 7f051159a700 1 -- 172.21.15.62:0/355053045 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05040042c0 con 0x7f0514105b40 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.948+0000 7f051159a700 1 -- 172.21.15.62:0/355053045 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05040119b0 con 0x7f0514105b40 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.948+0000 7f0518963700 1 -- 172.21.15.62:0/355053045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 msgr2=0x7f0514105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.948+0000 7f0518963700 1 --2- 172.21.15.62:0/355053045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f0514105f60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05040098a0 tx=0x7f0504009bb0 comp rx=0 tx=0).stop 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.949+0000 7f0518963700 1 -- 172.21.15.62:0/355053045 shutdown_connections 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.949+0000 7f0518963700 1 --2- 172.21.15.62:0/355053045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f0514105f60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.949+0000 7f0518963700 1 -- 172.21.15.62:0/355053045 >> 172.21.15.62:0/355053045 conn(0x7f0514100340 msgr2=0x7f05141027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.949+0000 7f0518963700 1 -- 172.21.15.62:0/355053045 shutdown_connections 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.949+0000 7f0518963700 1 -- 172.21.15.62:0/355053045 wait complete. 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.949+0000 7f0518963700 1 Processor -- start 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.949+0000 7f0518963700 1 -- start start 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f0518963700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f05141906f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:08.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f0518963700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0514190c30 con 0x7f0514105b40 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f051259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f05141906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f051259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f05141906f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50630/0 (socket says 172.21.15.62:50630) 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f051259c700 1 -- 172.21.15.62:0/1520305278 learned_addr learned my addr 172.21.15.62:0/1520305278 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f051259c700 1 -- 172.21.15.62:0/1520305278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0504009550 con 0x7f0514105b40 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f051259c700 1 --2- 172.21.15.62:0/1520305278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f05141906f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0504006af0 tx=0x7f0504011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f05037fe700 1 -- 172.21.15.62:0/1520305278 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0504003a40 con 0x7f0514105b40 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f0518963700 1 -- 172.21.15.62:0/1520305278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051418ee70 con 0x7f0514105b40 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f05037fe700 1 -- 172.21.15.62:0/1520305278 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0504003ba0 con 0x7f0514105b40 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f05037fe700 1 -- 172.21.15.62:0/1520305278 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05040196c0 con 0x7f0514105b40 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.950+0000 7f0518963700 1 -- 172.21.15.62:0/1520305278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051418f310 con 0x7f0514105b40 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.951+0000 7f05037fe700 1 -- 172.21.15.62:0/1520305278 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 11) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f0504019980 con 0x7f0514105b40 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.951+0000 7f05037fe700 1 --2- 172.21.15.62:0/1520305278 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f04fc038700 0x7f04fc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.951+0000 7f05037fe700 1 -- 172.21.15.62:0/1520305278 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f050404cc20 con 0x7f0514105b40 2024-05-08T22:30:08.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.952+0000 7f0511d9b700 1 --2- 172.21.15.62:0/1520305278 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f04fc038700 0x7f04fc03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.952+0000 7f0518963700 1 -- 172.21.15.62:0/1520305278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051418fd20 con 0x7f0514105b40 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.952+0000 7f0511d9b700 1 --2- 172.21.15.62:0/1520305278 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f04fc038700 0x7f04fc03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0508006fd0 tx=0x7f0508006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:07.957+0000 7f05037fe700 1 -- 172.21.15.62:0/1520305278 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f05040243f0 con 0x7f0514105b40 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.175+0000 7f0518963700 1 -- 172.21.15.62:0/1520305278 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f05140623f0 con 0x7f04fc038700 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.421+0000 7f05037fe700 1 -- 172.21.15.62:0/1520305278 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f05140623f0 con 0x7f04fc038700 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.423+0000 7f0518963700 1 -- 172.21.15.62:0/1520305278 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f04fc038700 msgr2=0x7f04fc03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.423+0000 7f0518963700 1 --2- 172.21.15.62:0/1520305278 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f04fc038700 0x7f04fc03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0508006fd0 tx=0x7f0508006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.423+0000 7f0518963700 1 -- 172.21.15.62:0/1520305278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 msgr2=0x7f05141906f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.423+0000 7f0518963700 1 --2- 172.21.15.62:0/1520305278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f05141906f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0504006af0 tx=0x7f0504011a10 comp rx=0 tx=0).stop 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.423+0000 7f0518963700 1 -- 172.21.15.62:0/1520305278 shutdown_connections 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.423+0000 7f0518963700 1 --2- 172.21.15.62:0/1520305278 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f04fc038700 0x7f04fc03abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.423+0000 7f0518963700 1 --2- 172.21.15.62:0/1520305278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0514105b40 0x7f05141906f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:08.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.423+0000 7f0518963700 1 -- 172.21.15.62:0/1520305278 >> 172.21.15.62:0/1520305278 conn(0x7f0514100340 msgr2=0x7f0514101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:08.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.423+0000 7f0518963700 1 -- 172.21.15.62:0/1520305278 shutdown_connections 2024-05-08T22:30:08.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:08.423+0000 7f0518963700 1 -- 172.21.15.62:0/1520305278 wait complete. 2024-05-08T22:30:08.781 INFO:teuthology.orchestra.run.smithi062.stdout:Fetching dashboard port number... 2024-05-08T22:30:09.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:09 smithi062 ceph-mon[42571]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:09.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:09.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:09 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1601957982' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T22:30:09.685 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 8443 2024-05-08T22:30:09.685 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.110+0000 7f3e7ecd8700 1 Processor -- start 2024-05-08T22:30:09.685 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.110+0000 7f3e7ecd8700 1 -- start start 2024-05-08T22:30:09.685 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.110+0000 7f3e7ecd8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e78109570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.110+0000 7f3e7ecd8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e78079eb0 con 0x7f3e78107180 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.110+0000 7f3e7ca74700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e78109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.110+0000 7f3e7ca74700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e78109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50640/0 (socket says 172.21.15.62:50640) 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.110+0000 7f3e7ca74700 1 -- 172.21.15.62:0/2820278804 learned_addr learned my addr 172.21.15.62:0/2820278804 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.111+0000 7f3e7ca74700 1 -- 172.21.15.62:0/2820278804 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e78109ab0 con 0x7f3e78107180 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.111+0000 7f3e7ca74700 1 --2- 172.21.15.62:0/2820278804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e78109570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0098a0 tx=0x7f3e6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e6d847b5691dff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.111+0000 7f3e777fe700 1 -- 172.21.15.62:0/2820278804 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e6c004160 con 0x7f3e78107180 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.111+0000 7f3e777fe700 1 -- 172.21.15.62:0/2820278804 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e6c0042c0 con 0x7f3e78107180 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.111+0000 7f3e777fe700 1 -- 172.21.15.62:0/2820278804 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e6c0119b0 con 0x7f3e78107180 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.112+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/2820278804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 msgr2=0x7f3e78109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.112+0000 7f3e7ecd8700 1 --2- 172.21.15.62:0/2820278804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e78109570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0098a0 tx=0x7f3e6c009bb0 comp rx=0 tx=0).stop 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.112+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/2820278804 shutdown_connections 2024-05-08T22:30:09.686 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.112+0000 7f3e7ecd8700 1 --2- 172.21.15.62:0/2820278804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e78109570 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.112+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/2820278804 >> 172.21.15.62:0/2820278804 conn(0x7f3e78100bb0 msgr2=0x7f3e78102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.112+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/2820278804 shutdown_connections 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.112+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/2820278804 wait complete. 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.112+0000 7f3e7ecd8700 1 Processor -- start 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.112+0000 7f3e7ecd8700 1 -- start start 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.113+0000 7f3e7ecd8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e7819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.113+0000 7f3e7ecd8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7819a9a0 con 0x7f3e78107180 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.113+0000 7f3e7ca74700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e7819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.113+0000 7f3e7ca74700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e7819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50642/0 (socket says 172.21.15.62:50642) 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.113+0000 7f3e7ca74700 1 -- 172.21.15.62:0/1601957982 learned_addr learned my addr 172.21.15.62:0/1601957982 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.113+0000 7f3e7ca74700 1 -- 172.21.15.62:0/1601957982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c009550 con 0x7f3e78107180 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.113+0000 7f3e7ca74700 1 --2- 172.21.15.62:0/1601957982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e7819a460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c011670 tx=0x7f3e6c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.113+0000 7f3e75ffb700 1 -- 172.21.15.62:0/1601957982 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e6c003b20 con 0x7f3e78107180 2024-05-08T22:30:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.113+0000 7f3e75ffb700 1 -- 172.21.15.62:0/1601957982 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e6c003c80 con 0x7f3e78107180 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.113+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/1601957982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7819aba0 con 0x7f3e78107180 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.113+0000 7f3e75ffb700 1 -- 172.21.15.62:0/1601957982 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e6c019770 con 0x7f3e78107180 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.114+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/1601957982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7819b040 con 0x7f3e78107180 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.114+0000 7f3e75ffb700 1 -- 172.21.15.62:0/1601957982 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 11) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f3e6c019a30 con 0x7f3e78107180 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.114+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/1601957982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7810a8f0 con 0x7f3e78107180 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.115+0000 7f3e75ffb700 1 --2- 172.21.15.62:0/1601957982 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3e68038750 0x7f3e6803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.115+0000 7f3e75ffb700 1 -- 172.21.15.62:0/1601957982 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3e6c04cac0 con 0x7f3e78107180 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.115+0000 7f3e77fff700 1 --2- 172.21.15.62:0/1601957982 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3e68038750 0x7f3e6803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.115+0000 7f3e77fff700 1 --2- 172.21.15.62:0/1601957982 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3e68038750 0x7f3e6803ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e64006fd0 tx=0x7f3e64006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.119+0000 7f3e75ffb700 1 -- 172.21.15.62:0/1601957982 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3e6c018db0 con 0x7f3e78107180 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.329+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/1601957982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f3e7819d8d0 con 0x7f3e78107180 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.330+0000 7f3e75ffb700 1 -- 172.21.15.62:0/1601957982 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f3e6c04ce60 con 0x7f3e78107180 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.332+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/1601957982 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3e68038750 msgr2=0x7f3e6803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:09.688 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.332+0000 7f3e7ecd8700 1 --2- 172.21.15.62:0/1601957982 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3e68038750 0x7f3e6803ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e64006fd0 tx=0x7f3e64006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:09.689 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.332+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/1601957982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 msgr2=0x7f3e7819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:09.689 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.332+0000 7f3e7ecd8700 1 --2- 172.21.15.62:0/1601957982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e7819a460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c011670 tx=0x7f3e6c011a30 comp rx=0 tx=0).stop 2024-05-08T22:30:09.689 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.332+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/1601957982 shutdown_connections 2024-05-08T22:30:09.689 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.332+0000 7f3e7ecd8700 1 --2- 172.21.15.62:0/1601957982 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3e68038750 0x7f3e6803ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:09.689 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.332+0000 7f3e7ecd8700 1 --2- 172.21.15.62:0/1601957982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e78107180 0x7f3e7819a460 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:09.689 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.332+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/1601957982 >> 172.21.15.62:0/1601957982 conn(0x7f3e78100bb0 msgr2=0x7f3e781017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:09.689 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.332+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/1601957982 shutdown_connections 2024-05-08T22:30:09.689 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:09.332+0000 7f3e7ecd8700 1 -- 172.21.15.62:0/1601957982 wait complete. 2024-05-08T22:30:09.693 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T22:30:09.693 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-05-08T22:30:09.701 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T22:30:09.701 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-05-08T22:30:09.701 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-05-08T22:30:09.701 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-08T22:30:09.702 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph Dashboard is now available at: 2024-05-08T22:30:09.702 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:09.702 INFO:teuthology.orchestra.run.smithi062.stdout: URL: https://smithi062:8443/ 2024-05-08T22:30:09.702 INFO:teuthology.orchestra.run.smithi062.stdout: User: admin 2024-05-08T22:30:09.702 INFO:teuthology.orchestra.run.smithi062.stdout: Password: i16i2f24oi 2024-05-08T22:30:09.702 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:09.703 INFO:teuthology.orchestra.run.smithi062.stdout:Saving cluster configuration to /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config directory 2024-05-08T22:30:09.703 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling autotune for osd_memory_target 2024-05-08T22:30:10.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.077+0000 7f2af47e0700 1 Processor -- start 2024-05-08T22:30:10.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.077+0000 7f2af47e0700 1 -- start start 2024-05-08T22:30:10.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.077+0000 7f2af47e0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec105a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:10.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.078+0000 7f2af47e0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec106030 con 0x7f2aec105640 2024-05-08T22:30:10.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.078+0000 7f2af257c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec105a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:10.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.078+0000 7f2af257c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec105a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50652/0 (socket says 172.21.15.62:50652) 2024-05-08T22:30:10.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.078+0000 7f2af257c700 1 -- 172.21.15.62:0/32842108 learned_addr learned my addr 172.21.15.62:0/32842108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:10.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.078+0000 7f2af257c700 1 -- 172.21.15.62:0/32842108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec106840 con 0x7f2aec105640 2024-05-08T22:30:10.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.078+0000 7f2af257c700 1 --2- 172.21.15.62:0/32842108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec105a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0098a0 tx=0x7f2adc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7bc3dc7d00d00e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:10.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.078+0000 7f2af157a700 1 -- 172.21.15.62:0/32842108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2adc004160 con 0x7f2aec105640 2024-05-08T22:30:10.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.078+0000 7f2af157a700 1 -- 172.21.15.62:0/32842108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2adc0042c0 con 0x7f2aec105640 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.078+0000 7f2af157a700 1 -- 172.21.15.62:0/32842108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2adc0119b0 con 0x7f2aec105640 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.079+0000 7f2af47e0700 1 -- 172.21.15.62:0/32842108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 msgr2=0x7f2aec105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.079+0000 7f2af47e0700 1 --2- 172.21.15.62:0/32842108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec105a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0098a0 tx=0x7f2adc009bb0 comp rx=0 tx=0).stop 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.079+0000 7f2af47e0700 1 -- 172.21.15.62:0/32842108 shutdown_connections 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.079+0000 7f2af47e0700 1 --2- 172.21.15.62:0/32842108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec105a60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.079+0000 7f2af47e0700 1 -- 172.21.15.62:0/32842108 >> 172.21.15.62:0/32842108 conn(0x7f2aec100ba0 msgr2=0x7f2aec103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.079+0000 7f2af47e0700 1 -- 172.21.15.62:0/32842108 shutdown_connections 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.079+0000 7f2af47e0700 1 -- 172.21.15.62:0/32842108 wait complete. 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2af47e0700 1 Processor -- start 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2af47e0700 1 -- start start 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2af47e0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2af47e0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec19aa20 con 0x7f2aec105640 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2af257c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:10.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2af257c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50664/0 (socket says 172.21.15.62:50664) 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2af257c700 1 -- 172.21.15.62:0/135875907 learned_addr learned my addr 172.21.15.62:0/135875907 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2af257c700 1 -- 172.21.15.62:0/135875907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc009550 con 0x7f2aec105640 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2af257c700 1 --2- 172.21.15.62:0/135875907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec19a4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2adc003e60 tx=0x7f2adc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2ae37fe700 1 -- 172.21.15.62:0/135875907 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2adc003a60 con 0x7f2aec105640 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2ae37fe700 1 -- 172.21.15.62:0/135875907 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2adc003bc0 con 0x7f2aec105640 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2ae37fe700 1 -- 172.21.15.62:0/135875907 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2adc0196c0 con 0x7f2aec105640 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.080+0000 7f2af47e0700 1 -- 172.21.15.62:0/135875907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec19ac20 con 0x7f2aec105640 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.081+0000 7f2af47e0700 1 -- 172.21.15.62:0/135875907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec19b0c0 con 0x7f2aec105640 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.081+0000 7f2ae37fe700 1 -- 172.21.15.62:0/135875907 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 11) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f2adc01ab60 con 0x7f2aec105640 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.081+0000 7f2af47e0700 1 -- 172.21.15.62:0/135875907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec1946e0 con 0x7f2aec105640 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.082+0000 7f2ae37fe700 1 --2- 172.21.15.62:0/135875907 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2ad8038780 0x7f2ad803ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.082+0000 7f2ae37fe700 1 -- 172.21.15.62:0/135875907 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2adc04ca30 con 0x7f2aec105640 2024-05-08T22:30:10.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.082+0000 7f2af1d7b700 1 --2- 172.21.15.62:0/135875907 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2ad8038780 0x7f2ad803ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.082+0000 7f2af1d7b700 1 --2- 172.21.15.62:0/135875907 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2ad8038780 0x7f2ad803ac40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8006fd0 tx=0x7f2ae8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.086+0000 7f2ae37fe700 1 -- 172.21.15.62:0/135875907 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2adc018b00 con 0x7f2aec105640 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.287+0000 7f2af47e0700 1 -- 172.21.15.62:0/135875907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f2aec109c40 con 0x7f2aec105640 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.287+0000 7f2ae37fe700 1 -- 172.21.15.62:0/135875907 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f2adc018b00 con 0x7f2aec105640 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.290+0000 7f2af47e0700 1 -- 172.21.15.62:0/135875907 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2ad8038780 msgr2=0x7f2ad803ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.290+0000 7f2af47e0700 1 --2- 172.21.15.62:0/135875907 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2ad8038780 0x7f2ad803ac40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8006fd0 tx=0x7f2ae8006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.290+0000 7f2af47e0700 1 -- 172.21.15.62:0/135875907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 msgr2=0x7f2aec19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.290+0000 7f2af47e0700 1 --2- 172.21.15.62:0/135875907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec19a4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2adc003e60 tx=0x7f2adc011a30 comp rx=0 tx=0).stop 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.290+0000 7f2af47e0700 1 -- 172.21.15.62:0/135875907 shutdown_connections 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.290+0000 7f2af47e0700 1 --2- 172.21.15.62:0/135875907 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2ad8038780 0x7f2ad803ac40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.290+0000 7f2af47e0700 1 --2- 172.21.15.62:0/135875907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105640 0x7f2aec19a4e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.290+0000 7f2af47e0700 1 -- 172.21.15.62:0/135875907 >> 172.21.15.62:0/135875907 conn(0x7f2aec100ba0 msgr2=0x7f2aec101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.290+0000 7f2af47e0700 1 -- 172.21.15.62:0/135875907 shutdown_connections 2024-05-08T22:30:10.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.290+0000 7f2af47e0700 1 -- 172.21.15.62:0/135875907 wait complete. 2024-05-08T22:30:11.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.990+0000 7f135a9a0700 1 Processor -- start 2024-05-08T22:30:11.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.990+0000 7f135a9a0700 1 -- start start 2024-05-08T22:30:11.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.990+0000 7f135a9a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f13541052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.991+0000 7f135a9a0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1354074820 con 0x7f1354104e90 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.991+0000 7f135999e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f13541052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.991+0000 7f135999e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f13541052b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50674/0 (socket says 172.21.15.62:50674) 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.991+0000 7f135999e700 1 -- 172.21.15.62:0/589166518 learned_addr learned my addr 172.21.15.62:0/589166518 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.991+0000 7f135999e700 1 -- 172.21.15.62:0/589166518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13541057f0 con 0x7f1354104e90 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.991+0000 7f135999e700 1 --2- 172.21.15.62:0/589166518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f13541052b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13440098a0 tx=0x7f1344009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5c6617daf72d0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.991+0000 7f135899c700 1 -- 172.21.15.62:0/589166518 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1344004160 con 0x7f1354104e90 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.991+0000 7f135899c700 1 -- 172.21.15.62:0/589166518 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13440042c0 con 0x7f1354104e90 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.991+0000 7f135899c700 1 -- 172.21.15.62:0/589166518 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13440119b0 con 0x7f1354104e90 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.992+0000 7f135a9a0700 1 -- 172.21.15.62:0/589166518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 msgr2=0x7f13541052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.992+0000 7f135a9a0700 1 --2- 172.21.15.62:0/589166518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f13541052b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13440098a0 tx=0x7f1344009bb0 comp rx=0 tx=0).stop 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.992+0000 7f135a9a0700 1 -- 172.21.15.62:0/589166518 shutdown_connections 2024-05-08T22:30:11.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.992+0000 7f135a9a0700 1 --2- 172.21.15.62:0/589166518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f13541052b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.992+0000 7f135a9a0700 1 -- 172.21.15.62:0/589166518 >> 172.21.15.62:0/589166518 conn(0x7f1354100ab0 msgr2=0x7f1354102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.992+0000 7f135a9a0700 1 -- 172.21.15.62:0/589166518 shutdown_connections 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.992+0000 7f135a9a0700 1 -- 172.21.15.62:0/589166518 wait complete. 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.993+0000 7f135a9a0700 1 Processor -- start 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.993+0000 7f135a9a0700 1 -- start start 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.993+0000 7f135a9a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f135419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.993+0000 7f135a9a0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135419cb10 con 0x7f1354104e90 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.993+0000 7f135999e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f135419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.993+0000 7f135999e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f135419c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50678/0 (socket says 172.21.15.62:50678) 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.993+0000 7f135999e700 1 -- 172.21.15.62:0/3517234272 learned_addr learned my addr 172.21.15.62:0/3517234272 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.993+0000 7f135999e700 1 -- 172.21.15.62:0/3517234272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1344009550 con 0x7f1354104e90 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.994+0000 7f135999e700 1 --2- 172.21.15.62:0/3517234272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f135419c5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1344003e60 tx=0x7f1344011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.994+0000 7f134affd700 1 -- 172.21.15.62:0/3517234272 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13440038b0 con 0x7f1354104e90 2024-05-08T22:30:11.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.994+0000 7f135a9a0700 1 -- 172.21.15.62:0/3517234272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135419cd10 con 0x7f1354104e90 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.994+0000 7f134affd700 1 -- 172.21.15.62:0/3517234272 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1344003a10 con 0x7f1354104e90 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.994+0000 7f134affd700 1 -- 172.21.15.62:0/3517234272 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1344019530 con 0x7f1354104e90 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.994+0000 7f135a9a0700 1 -- 172.21.15.62:0/3517234272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135419d1b0 con 0x7f1354104e90 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.995+0000 7f134affd700 1 -- 172.21.15.62:0/3517234272 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f134401a980 con 0x7f1354104e90 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.995+0000 7f135a9a0700 1 -- 172.21.15.62:0/3517234272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13541962a0 con 0x7f1354104e90 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.995+0000 7f134affd700 1 --2- 172.21.15.62:0/3517234272 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f13400387a0 0x7f134003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.995+0000 7f134affd700 1 -- 172.21.15.62:0/3517234272 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f134404ca80 con 0x7f1354104e90 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.995+0000 7f135919d700 1 --2- 172.21.15.62:0/3517234272 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f13400387a0 0x7f134003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:10.995+0000 7f135919d700 1 --2- 172.21.15.62:0/3517234272 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f13400387a0 0x7f134003ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1350006fd0 tx=0x7f1350006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.000+0000 7f134affd700 1 -- 172.21.15.62:0/3517234272 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1344003b80 con 0x7f1354104e90 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.314+0000 7f135a9a0700 1 -- 172.21.15.62:0/3517234272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f135402cea0 con 0x7f1354104e90 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.316+0000 7f134affd700 1 -- 172.21.15.62:0/3517234272 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f134404c310 con 0x7f1354104e90 2024-05-08T22:30:11.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.318+0000 7f135a9a0700 1 -- 172.21.15.62:0/3517234272 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f13400387a0 msgr2=0x7f134003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.318+0000 7f135a9a0700 1 --2- 172.21.15.62:0/3517234272 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f13400387a0 0x7f134003ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1350006fd0 tx=0x7f1350006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.318+0000 7f135a9a0700 1 -- 172.21.15.62:0/3517234272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 msgr2=0x7f135419c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.318+0000 7f135a9a0700 1 --2- 172.21.15.62:0/3517234272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f135419c5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1344003e60 tx=0x7f1344011a30 comp rx=0 tx=0).stop 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.318+0000 7f135a9a0700 1 -- 172.21.15.62:0/3517234272 shutdown_connections 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.318+0000 7f135a9a0700 1 --2- 172.21.15.62:0/3517234272 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f13400387a0 0x7f134003ac60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.318+0000 7f135a9a0700 1 --2- 172.21.15.62:0/3517234272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1354104e90 0x7f135419c5d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.318+0000 7f135a9a0700 1 -- 172.21.15.62:0/3517234272 >> 172.21.15.62:0/3517234272 conn(0x7f1354100ab0 msgr2=0x7f13541933e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.319+0000 7f135a9a0700 1 -- 172.21.15.62:0/3517234272 shutdown_connections 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-05-08T22:30:11.319+0000 7f135a9a0700 1 -- 172.21.15.62:0/3517234272 wait complete. 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout:Or, if you are only running a single cluster on this host: 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: ceph telemetry on 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout:For more information see: 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:11.673 INFO:teuthology.orchestra.run.smithi062.stdout:Bootstrap complete. 2024-05-08T22:30:11.674 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:11 smithi062 ceph-mon[42571]: mgrmap e12: x(active, since 6s) 2024-05-08T22:30:11.674 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:11 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3517234272' entity='client.admin' 2024-05-08T22:30:11.702 INFO:tasks.cephadm:Fetching config... 2024-05-08T22:30:11.702 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:30:11.702 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-08T22:30:11.720 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-08T22:30:11.720 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:30:11.720 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-08T22:30:11.778 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-08T22:30:11.778 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:30:11.778 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-08T22:30:11.846 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-08T22:30:11.847 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:30:11.847 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-08T22:30:11.906 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-08T22:30:11.907 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWOjB9Ltp5gYBnuRoFQVO9TKYFzYiYiBE0iP9I/NGPjHm5ap7zGthnvqE+ExhRuQ3+S03RLp/hZ45Wsyu31/EJP+6Q+vuSgWwmala7eRQVOgjM6jnvXKbFFfTw7lcuw6TZQjXE6oVV8dLM5FkF+e9cOj46adtdEgI0QgVqrZ8Gf3YfsIu1PtH7YPScaRt5/jQFz+Lav3Dxr9btSriMHAZ+7AIRkn5qoQh6kFiopsrGFTm0tDn373h8k9t9zwQx8ZkjxZ6WC9MZKJipIWoPnOekJmoDxMKH4fUPmRKBio7rg8F57Oyi+HBP50BFyPlRKjOIehIUWbErJFRfWWFquGE55shmOzOsuTKqDhu6wS9giVyI8Vcg2xWnLecpyGvVQ7t+nVVfehyQ/ATXhIkSnUYGz1aXzHOqTrRRJHdVLOK7UyvpAq6xsdLVnhqw5SdJsCBOUd1JqdopV3WXN8qQCjEWc2/goo9qHZVaiJQchFhdrAja7f9RHjAfumF/ITWvtR0= ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T22:30:11.988 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWOjB9Ltp5gYBnuRoFQVO9TKYFzYiYiBE0iP9I/NGPjHm5ap7zGthnvqE+ExhRuQ3+S03RLp/hZ45Wsyu31/EJP+6Q+vuSgWwmala7eRQVOgjM6jnvXKbFFfTw7lcuw6TZQjXE6oVV8dLM5FkF+e9cOj46adtdEgI0QgVqrZ8Gf3YfsIu1PtH7YPScaRt5/jQFz+Lav3Dxr9btSriMHAZ+7AIRkn5qoQh6kFiopsrGFTm0tDn373h8k9t9zwQx8ZkjxZ6WC9MZKJipIWoPnOekJmoDxMKH4fUPmRKBio7rg8F57Oyi+HBP50BFyPlRKjOIehIUWbErJFRfWWFquGE55shmOzOsuTKqDhu6wS9giVyI8Vcg2xWnLecpyGvVQ7t+nVVfehyQ/ATXhIkSnUYGz1aXzHOqTrRRJHdVLOK7UyvpAq6xsdLVnhqw5SdJsCBOUd1JqdopV3WXN8qQCjEWc2/goo9qHZVaiJQchFhdrAja7f9RHjAfumF/ITWvtR0= ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:30:12.001 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWOjB9Ltp5gYBnuRoFQVO9TKYFzYiYiBE0iP9I/NGPjHm5ap7zGthnvqE+ExhRuQ3+S03RLp/hZ45Wsyu31/EJP+6Q+vuSgWwmala7eRQVOgjM6jnvXKbFFfTw7lcuw6TZQjXE6oVV8dLM5FkF+e9cOj46adtdEgI0QgVqrZ8Gf3YfsIu1PtH7YPScaRt5/jQFz+Lav3Dxr9btSriMHAZ+7AIRkn5qoQh6kFiopsrGFTm0tDn373h8k9t9zwQx8ZkjxZ6WC9MZKJipIWoPnOekJmoDxMKH4fUPmRKBio7rg8F57Oyi+HBP50BFyPlRKjOIehIUWbErJFRfWWFquGE55shmOzOsuTKqDhu6wS9giVyI8Vcg2xWnLecpyGvVQ7t+nVVfehyQ/ATXhIkSnUYGz1aXzHOqTrRRJHdVLOK7UyvpAq6xsdLVnhqw5SdJsCBOUd1JqdopV3WXN8qQCjEWc2/goo9qHZVaiJQchFhdrAja7f9RHjAfumF/ITWvtR0= ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T22:30:12.052 INFO:teuthology.orchestra.run.smithi066.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWOjB9Ltp5gYBnuRoFQVO9TKYFzYiYiBE0iP9I/NGPjHm5ap7zGthnvqE+ExhRuQ3+S03RLp/hZ45Wsyu31/EJP+6Q+vuSgWwmala7eRQVOgjM6jnvXKbFFfTw7lcuw6TZQjXE6oVV8dLM5FkF+e9cOj46adtdEgI0QgVqrZ8Gf3YfsIu1PtH7YPScaRt5/jQFz+Lav3Dxr9btSriMHAZ+7AIRkn5qoQh6kFiopsrGFTm0tDn373h8k9t9zwQx8ZkjxZ6WC9MZKJipIWoPnOekJmoDxMKH4fUPmRKBio7rg8F57Oyi+HBP50BFyPlRKjOIehIUWbErJFRfWWFquGE55shmOzOsuTKqDhu6wS9giVyI8Vcg2xWnLecpyGvVQ7t+nVVfehyQ/ATXhIkSnUYGz1aXzHOqTrRRJHdVLOK7UyvpAq6xsdLVnhqw5SdJsCBOUd1JqdopV3WXN8qQCjEWc2/goo9qHZVaiJQchFhdrAja7f9RHjAfumF/ITWvtR0= ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:30:12.065 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWOjB9Ltp5gYBnuRoFQVO9TKYFzYiYiBE0iP9I/NGPjHm5ap7zGthnvqE+ExhRuQ3+S03RLp/hZ45Wsyu31/EJP+6Q+vuSgWwmala7eRQVOgjM6jnvXKbFFfTw7lcuw6TZQjXE6oVV8dLM5FkF+e9cOj46adtdEgI0QgVqrZ8Gf3YfsIu1PtH7YPScaRt5/jQFz+Lav3Dxr9btSriMHAZ+7AIRkn5qoQh6kFiopsrGFTm0tDn373h8k9t9zwQx8ZkjxZ6WC9MZKJipIWoPnOekJmoDxMKH4fUPmRKBio7rg8F57Oyi+HBP50BFyPlRKjOIehIUWbErJFRfWWFquGE55shmOzOsuTKqDhu6wS9giVyI8Vcg2xWnLecpyGvVQ7t+nVVfehyQ/ATXhIkSnUYGz1aXzHOqTrRRJHdVLOK7UyvpAq6xsdLVnhqw5SdJsCBOUd1JqdopV3WXN8qQCjEWc2/goo9qHZVaiJQchFhdrAja7f9RHjAfumF/ITWvtR0= ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T22:30:12.112 INFO:teuthology.orchestra.run.smithi079.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWOjB9Ltp5gYBnuRoFQVO9TKYFzYiYiBE0iP9I/NGPjHm5ap7zGthnvqE+ExhRuQ3+S03RLp/hZ45Wsyu31/EJP+6Q+vuSgWwmala7eRQVOgjM6jnvXKbFFfTw7lcuw6TZQjXE6oVV8dLM5FkF+e9cOj46adtdEgI0QgVqrZ8Gf3YfsIu1PtH7YPScaRt5/jQFz+Lav3Dxr9btSriMHAZ+7AIRkn5qoQh6kFiopsrGFTm0tDn373h8k9t9zwQx8ZkjxZ6WC9MZKJipIWoPnOekJmoDxMKH4fUPmRKBio7rg8F57Oyi+HBP50BFyPlRKjOIehIUWbErJFRfWWFquGE55shmOzOsuTKqDhu6wS9giVyI8Vcg2xWnLecpyGvVQ7t+nVVfehyQ/ATXhIkSnUYGz1aXzHOqTrRRJHdVLOK7UyvpAq6xsdLVnhqw5SdJsCBOUd1JqdopV3WXN8qQCjEWc2/goo9qHZVaiJQchFhdrAja7f9RHjAfumF/ITWvtR0= ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:30:12.125 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-08T22:30:12.353 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:30:13.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.530+0000 7f8d97fbf700 1 -- 172.21.15.62:0/3121946879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d900ff830 msgr2=0x7f8d900ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:13.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.530+0000 7f8d97fbf700 1 --2- 172.21.15.62:0/3121946879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d900ff830 0x7f8d900ffc50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009910 tx=0x7f8d8c009c20 comp rx=0 tx=0).stop 2024-05-08T22:30:13.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.530+0000 7f8d97fbf700 1 -- 172.21.15.62:0/3121946879 shutdown_connections 2024-05-08T22:30:13.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.530+0000 7f8d97fbf700 1 --2- 172.21.15.62:0/3121946879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d900ff830 0x7f8d900ffc50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:13.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.530+0000 7f8d97fbf700 1 -- 172.21.15.62:0/3121946879 >> 172.21.15.62:0/3121946879 conn(0x7f8d900fadd0 msgr2=0x7f8d900fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:13.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.530+0000 7f8d97fbf700 1 -- 172.21.15.62:0/3121946879 shutdown_connections 2024-05-08T22:30:13.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.530+0000 7f8d97fbf700 1 -- 172.21.15.62:0/3121946879 wait complete. 2024-05-08T22:30:13.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.531+0000 7f8d97fbf700 1 Processor -- start 2024-05-08T22:30:13.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.531+0000 7f8d97fbf700 1 -- start start 2024-05-08T22:30:13.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.531+0000 7f8d97fbf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d900ff830 0x7f8d90197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:13.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.531+0000 7f8d97fbf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d90198500 con 0x7f8d900ff830 2024-05-08T22:30:13.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.531+0000 7f8d95d5b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d900ff830 0x7f8d90197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:13.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.531+0000 7f8d95d5b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d900ff830 0x7f8d90197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50686/0 (socket says 172.21.15.62:50686) 2024-05-08T22:30:13.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.531+0000 7f8d95d5b700 1 -- 172.21.15.62:0/2884077148 learned_addr learned my addr 172.21.15.62:0/2884077148 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:13.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.531+0000 7f8d95d5b700 1 -- 172.21.15.62:0/2884077148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c0095f0 con 0x7f8d900ff830 2024-05-08T22:30:13.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.531+0000 7f8d95d5b700 1 --2- 172.21.15.62:0/2884077148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d900ff830 0x7f8d90197fc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c005950 tx=0x7f8d8c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:13.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.531+0000 7f8d86ffd700 1 -- 172.21.15.62:0/2884077148 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d8c01d070 con 0x7f8d900ff830 2024-05-08T22:30:13.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.532+0000 7f8d86ffd700 1 -- 172.21.15.62:0/2884077148 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d8c00bc10 con 0x7f8d900ff830 2024-05-08T22:30:13.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.532+0000 7f8d97fbf700 1 -- 172.21.15.62:0/2884077148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d90198700 con 0x7f8d900ff830 2024-05-08T22:30:13.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.532+0000 7f8d86ffd700 1 -- 172.21.15.62:0/2884077148 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d8c003a10 con 0x7f8d900ff830 2024-05-08T22:30:13.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.532+0000 7f8d97fbf700 1 -- 172.21.15.62:0/2884077148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d90198ba0 con 0x7f8d900ff830 2024-05-08T22:30:13.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.532+0000 7f8d97fbf700 1 -- 172.21.15.62:0/2884077148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d90192070 con 0x7f8d900ff830 2024-05-08T22:30:13.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.533+0000 7f8d86ffd700 1 -- 172.21.15.62:0/2884077148 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f8d8c003c80 con 0x7f8d900ff830 2024-05-08T22:30:13.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.533+0000 7f8d86ffd700 1 --2- 172.21.15.62:0/2884077148 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8d7c0387f0 0x7f8d7c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:13.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.533+0000 7f8d86ffd700 1 -- 172.21.15.62:0/2884077148 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8d8c04c460 con 0x7f8d900ff830 2024-05-08T22:30:13.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.536+0000 7f8d9555a700 1 --2- 172.21.15.62:0/2884077148 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8d7c0387f0 0x7f8d7c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:13.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.536+0000 7f8d9555a700 1 --2- 172.21.15.62:0/2884077148 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8d7c0387f0 0x7f8d7c03acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d80006fd0 tx=0x7f8d80006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:13.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.537+0000 7f8d86ffd700 1 -- 172.21.15.62:0/2884077148 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8d8c00ba70 con 0x7f8d900ff830 2024-05-08T22:30:13.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.743+0000 7f8d97fbf700 1 -- 172.21.15.62:0/2884077148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f8d900625d0 con 0x7f8d900ff830 2024-05-08T22:30:13.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.746+0000 7f8d86ffd700 1 -- 172.21.15.62:0/2884077148 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8d900625d0 con 0x7f8d900ff830 2024-05-08T22:30:13.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.748+0000 7f8d97fbf700 1 -- 172.21.15.62:0/2884077148 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8d7c0387f0 msgr2=0x7f8d7c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:13.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.748+0000 7f8d97fbf700 1 --2- 172.21.15.62:0/2884077148 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8d7c0387f0 0x7f8d7c03acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d80006fd0 tx=0x7f8d80006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:13.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.748+0000 7f8d97fbf700 1 -- 172.21.15.62:0/2884077148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d900ff830 msgr2=0x7f8d90197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:13.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.748+0000 7f8d97fbf700 1 --2- 172.21.15.62:0/2884077148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d900ff830 0x7f8d90197fc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c005950 tx=0x7f8d8c0057c0 comp rx=0 tx=0).stop 2024-05-08T22:30:13.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.748+0000 7f8d97fbf700 1 -- 172.21.15.62:0/2884077148 shutdown_connections 2024-05-08T22:30:13.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.748+0000 7f8d97fbf700 1 --2- 172.21.15.62:0/2884077148 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8d7c0387f0 0x7f8d7c03acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:13.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.748+0000 7f8d97fbf700 1 --2- 172.21.15.62:0/2884077148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d900ff830 0x7f8d90197fc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:13.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.748+0000 7f8d97fbf700 1 -- 172.21.15.62:0/2884077148 >> 172.21.15.62:0/2884077148 conn(0x7f8d900fadd0 msgr2=0x7f8d9018f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:13.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.749+0000 7f8d97fbf700 1 -- 172.21.15.62:0/2884077148 shutdown_connections 2024-05-08T22:30:13.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:13.749+0000 7f8d97fbf700 1 -- 172.21.15.62:0/2884077148 wait complete. 2024-05-08T22:30:14.125 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-08T22:30:14.125 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-08T22:30:14.364 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:30:14.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:14.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:14.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:30:14.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:14.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:14.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:14.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:14 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2884077148' entity='client.admin' 2024-05-08T22:30:14.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:14.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:15.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.478+0000 7f627a9aa700 1 -- 172.21.15.62:0/2401789792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62740ff830 msgr2=0x7f62740ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:15.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.478+0000 7f627a9aa700 1 --2- 172.21.15.62:0/2401789792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62740ff830 0x7f62740ffc50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6264009910 tx=0x7f6264009c20 comp rx=0 tx=0).stop 2024-05-08T22:30:15.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.478+0000 7f627a9aa700 1 -- 172.21.15.62:0/2401789792 shutdown_connections 2024-05-08T22:30:15.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.478+0000 7f627a9aa700 1 --2- 172.21.15.62:0/2401789792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62740ff830 0x7f62740ffc50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:15.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.478+0000 7f627a9aa700 1 -- 172.21.15.62:0/2401789792 >> 172.21.15.62:0/2401789792 conn(0x7f62740fadd0 msgr2=0x7f62740fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:15.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.478+0000 7f627a9aa700 1 -- 172.21.15.62:0/2401789792 shutdown_connections 2024-05-08T22:30:15.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.478+0000 7f627a9aa700 1 -- 172.21.15.62:0/2401789792 wait complete. 2024-05-08T22:30:15.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.479+0000 7f627a9aa700 1 Processor -- start 2024-05-08T22:30:15.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.479+0000 7f627a9aa700 1 -- start start 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.479+0000 7f627a9aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62740ff830 0x7f6274194cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.479+0000 7f627a9aa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62741951f0 con 0x7f62740ff830 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.479+0000 7f6273fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62740ff830 0x7f6274194cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.479+0000 7f6273fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62740ff830 0x7f6274194cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50708/0 (socket says 172.21.15.62:50708) 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.479+0000 7f6273fff700 1 -- 172.21.15.62:0/108291866 learned_addr learned my addr 172.21.15.62:0/108291866 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.479+0000 7f6273fff700 1 -- 172.21.15.62:0/108291866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640095f0 con 0x7f62740ff830 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.479+0000 7f6273fff700 1 --2- 172.21.15.62:0/108291866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62740ff830 0x7f6274194cb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6264005950 tx=0x7f62640057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.480+0000 7f6271ffb700 1 -- 172.21.15.62:0/108291866 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f626401d070 con 0x7f62740ff830 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.480+0000 7f6271ffb700 1 -- 172.21.15.62:0/108291866 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f626400bc10 con 0x7f62740ff830 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.480+0000 7f627a9aa700 1 -- 172.21.15.62:0/108291866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62741953f0 con 0x7f62740ff830 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.480+0000 7f6271ffb700 1 -- 172.21.15.62:0/108291866 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6264003a10 con 0x7f62740ff830 2024-05-08T22:30:15.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.480+0000 7f627a9aa700 1 -- 172.21.15.62:0/108291866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6274195890 con 0x7f62740ff830 2024-05-08T22:30:15.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.481+0000 7f6271ffb700 1 -- 172.21.15.62:0/108291866 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f6264003c80 con 0x7f62740ff830 2024-05-08T22:30:15.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.481+0000 7f627a9aa700 1 -- 172.21.15.62:0/108291866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627418e550 con 0x7f62740ff830 2024-05-08T22:30:15.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.481+0000 7f6271ffb700 1 --2- 172.21.15.62:0/108291866 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f62600387f0 0x7f626003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:15.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.481+0000 7f6271ffb700 1 -- 172.21.15.62:0/108291866 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f626404c460 con 0x7f62740ff830 2024-05-08T22:30:15.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.481+0000 7f626bfff700 1 --2- 172.21.15.62:0/108291866 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f62600387f0 0x7f626003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:15.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.481+0000 7f626bfff700 1 --2- 172.21.15.62:0/108291866 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f62600387f0 0x7f626003acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f625c006fd0 tx=0x7f625c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:15.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.486+0000 7f6271ffb700 1 -- 172.21.15.62:0/108291866 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f626400ba70 con 0x7f62740ff830 2024-05-08T22:30:15.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.693+0000 7f627a9aa700 1 -- 172.21.15.62:0/108291866 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f62740613e0 con 0x7f62600387f0 2024-05-08T22:30:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.696+0000 7f6271ffb700 1 -- 172.21.15.62:0/108291866 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f62740613e0 con 0x7f62600387f0 2024-05-08T22:30:15.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.698+0000 7f627a9aa700 1 -- 172.21.15.62:0/108291866 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f62600387f0 msgr2=0x7f626003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.698+0000 7f627a9aa700 1 --2- 172.21.15.62:0/108291866 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f62600387f0 0x7f626003acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f625c006fd0 tx=0x7f625c006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.698+0000 7f627a9aa700 1 -- 172.21.15.62:0/108291866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62740ff830 msgr2=0x7f6274194cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.698+0000 7f627a9aa700 1 --2- 172.21.15.62:0/108291866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62740ff830 0x7f6274194cb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6264005950 tx=0x7f62640057c0 comp rx=0 tx=0).stop 2024-05-08T22:30:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.698+0000 7f627a9aa700 1 -- 172.21.15.62:0/108291866 shutdown_connections 2024-05-08T22:30:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.698+0000 7f627a9aa700 1 --2- 172.21.15.62:0/108291866 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f62600387f0 0x7f626003acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.698+0000 7f627a9aa700 1 --2- 172.21.15.62:0/108291866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62740ff830 0x7f6274194cb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.698+0000 7f627a9aa700 1 -- 172.21.15.62:0/108291866 >> 172.21.15.62:0/108291866 conn(0x7f62740fadd0 msgr2=0x7f62740fb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.699+0000 7f627a9aa700 1 -- 172.21.15.62:0/108291866 shutdown_connections 2024-05-08T22:30:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:15.699+0000 7f627a9aa700 1 -- 172.21.15.62:0/108291866 wait complete. 2024-05-08T22:30:16.075 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi066 2024-05-08T22:30:16.075 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-08T22:30:16.075 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/etc/ceph/ceph.conf 2024-05-08T22:30:16.097 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-08T22:30:16.097 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:16.157 INFO:tasks.cephadm:Adding host smithi066 to orchestrator... 2024-05-08T22:30:16.157 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch host add smithi066 2024-05-08T22:30:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:30:16.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:16 smithi062 ceph-mon[42571]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:16.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:16.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:16.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:16.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:16.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:16 smithi062 ceph-mon[42571]: Updating smithi062:/etc/ceph/ceph.conf 2024-05-08T22:30:17.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.376+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3940354925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01068b0 msgr2=0x7f93f0106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:17.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.376+0000 7f93f6c3f700 1 --2- 172.21.15.62:0/3940354925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01068b0 0x7f93f0106c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009910 tx=0x7f93e4009c20 comp rx=0 tx=0).stop 2024-05-08T22:30:17.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.376+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3940354925 shutdown_connections 2024-05-08T22:30:17.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.376+0000 7f93f6c3f700 1 --2- 172.21.15.62:0/3940354925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01068b0 0x7f93f0106c90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:17.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.376+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3940354925 >> 172.21.15.62:0/3940354925 conn(0x7f93f00fc3a0 msgr2=0x7f93f00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:17.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.376+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3940354925 shutdown_connections 2024-05-08T22:30:17.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.376+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3940354925 wait complete. 2024-05-08T22:30:17.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.377+0000 7f93f6c3f700 1 Processor -- start 2024-05-08T22:30:17.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.377+0000 7f93f6c3f700 1 -- start start 2024-05-08T22:30:17.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.377+0000 7f93f6c3f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01068b0 0x7f93f0196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:17.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.377+0000 7f93f6c3f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f0196690 con 0x7f93f01068b0 2024-05-08T22:30:17.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.377+0000 7f93f49db700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01068b0 0x7f93f0196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:17.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.377+0000 7f93f49db700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01068b0 0x7f93f0196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52684/0 (socket says 172.21.15.62:52684) 2024-05-08T22:30:17.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.377+0000 7f93f49db700 1 -- 172.21.15.62:0/3436289782 learned_addr learned my addr 172.21.15.62:0/3436289782 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:17.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.377+0000 7f93f49db700 1 -- 172.21.15.62:0/3436289782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40095f0 con 0x7f93f01068b0 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.378+0000 7f93f49db700 1 --2- 172.21.15.62:0/3436289782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01068b0 0x7f93f0196150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93e4004f40 tx=0x7f93e4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.378+0000 7f93edffb700 1 -- 172.21.15.62:0/3436289782 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93e401d070 con 0x7f93f01068b0 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.378+0000 7f93edffb700 1 -- 172.21.15.62:0/3436289782 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e400bbb0 con 0x7f93f01068b0 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.378+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3436289782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f01968f0 con 0x7f93f01068b0 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.378+0000 7f93edffb700 1 -- 172.21.15.62:0/3436289782 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93e4003810 con 0x7f93f01068b0 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.378+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3436289782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f0196d70 con 0x7f93f01068b0 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.379+0000 7f93edffb700 1 -- 172.21.15.62:0/3436289782 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f93e4003a40 con 0x7f93f01068b0 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.379+0000 7f93edffb700 1 --2- 172.21.15.62:0/3436289782 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f93e00387f0 0x7f93e003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.379+0000 7f93edffb700 1 -- 172.21.15.62:0/3436289782 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f93e404c400 con 0x7f93f01068b0 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.379+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3436289782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f0101ff0 con 0x7f93f01068b0 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.379+0000 7f93effff700 1 --2- 172.21.15.62:0/3436289782 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f93e00387f0 0x7f93e003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.379+0000 7f93effff700 1 --2- 172.21.15.62:0/3436289782 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f93e00387f0 0x7f93e003acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f93dc006fd0 tx=0x7f93dc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:17.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.384+0000 7f93edffb700 1 -- 172.21.15.62:0/3436289782 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f93e4026070 con 0x7f93f01068b0 2024-05-08T22:30:17.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:17.589+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3436289782 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi066", "target": ["mon-mgr", ""]}) v1 -- 0x7f93f0197500 con 0x7f93e00387f0 2024-05-08T22:30:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:17 smithi062 ceph-mon[42571]: Updating smithi062:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:17 smithi062 ceph-mon[42571]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:17 smithi062 ceph-mon[42571]: Updating smithi062:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:30:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:18.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:19.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:18 smithi062 ceph-mon[42571]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi066", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:19.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:19.909+0000 7f93edffb700 1 -- 172.21.15.62:0/3436289782 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f93f0197500 con 0x7f93e00387f0 2024-05-08T22:30:19.909 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi066' with addr '172.21.15.66' 2024-05-08T22:30:19.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:19.911+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3436289782 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f93e00387f0 msgr2=0x7f93e003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:19.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:19.911+0000 7f93f6c3f700 1 --2- 172.21.15.62:0/3436289782 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f93e00387f0 0x7f93e003acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f93dc006fd0 tx=0x7f93dc006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:19.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:19.911+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3436289782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01068b0 msgr2=0x7f93f0196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:19.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:19.911+0000 7f93f6c3f700 1 --2- 172.21.15.62:0/3436289782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01068b0 0x7f93f0196150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93e4004f40 tx=0x7f93e4005020 comp rx=0 tx=0).stop 2024-05-08T22:30:19.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:19.911+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3436289782 shutdown_connections 2024-05-08T22:30:19.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:19.911+0000 7f93f6c3f700 1 --2- 172.21.15.62:0/3436289782 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f93e00387f0 0x7f93e003acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:19.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:19.911+0000 7f93f6c3f700 1 --2- 172.21.15.62:0/3436289782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01068b0 0x7f93f0196150 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:19.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:19.911+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3436289782 >> 172.21.15.62:0/3436289782 conn(0x7f93f00fc3a0 msgr2=0x7f93f01084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:19.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:19.911+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3436289782 shutdown_connections 2024-05-08T22:30:19.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:19.911+0000 7f93f6c3f700 1 -- 172.21.15.62:0/3436289782 wait complete. 2024-05-08T22:30:20.069 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:19 smithi062 ceph-mon[42571]: Deploying cephadm binary to smithi066 2024-05-08T22:30:20.275 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T22:30:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:30:21.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:21.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:20 smithi062 ceph-mon[42571]: Added host smithi066 2024-05-08T22:30:21.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:21.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:21.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:21.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.440+0000 7faab7f73700 1 -- 172.21.15.62:0/1114962680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab0105820 msgr2=0x7faab0105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:21.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.440+0000 7faab7f73700 1 --2- 172.21.15.62:0/1114962680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab0105820 0x7faab0105c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faaac009910 tx=0x7faaac009c20 comp rx=0 tx=0).stop 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.440+0000 7faab7f73700 1 -- 172.21.15.62:0/1114962680 shutdown_connections 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.440+0000 7faab7f73700 1 --2- 172.21.15.62:0/1114962680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab0105820 0x7faab0105c00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.440+0000 7faab7f73700 1 -- 172.21.15.62:0/1114962680 >> 172.21.15.62:0/1114962680 conn(0x7faab0075350 msgr2=0x7faab0075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.440+0000 7faab7f73700 1 -- 172.21.15.62:0/1114962680 shutdown_connections 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.440+0000 7faab7f73700 1 -- 172.21.15.62:0/1114962680 wait complete. 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.441+0000 7faab7f73700 1 Processor -- start 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.441+0000 7faab7f73700 1 -- start start 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.441+0000 7faab7f73700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab0105820 0x7faab00fc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.441+0000 7faab7f73700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab00fe740 con 0x7faab0105820 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.441+0000 7faab5d0f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab0105820 0x7faab00fc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.441+0000 7faab5d0f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab0105820 0x7faab00fc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52688/0 (socket says 172.21.15.62:52688) 2024-05-08T22:30:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.441+0000 7faab5d0f700 1 -- 172.21.15.62:0/1171725258 learned_addr learned my addr 172.21.15.62:0/1171725258 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:21.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.442+0000 7faab5d0f700 1 -- 172.21.15.62:0/1171725258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac0095f0 con 0x7faab0105820 2024-05-08T22:30:21.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.442+0000 7faab5d0f700 1 --2- 172.21.15.62:0/1171725258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab0105820 0x7faab00fc880 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faaac006010 tx=0x7faaac004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:21.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.442+0000 7faaa2ffd700 1 -- 172.21.15.62:0/1171725258 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faaac01d070 con 0x7faab0105820 2024-05-08T22:30:21.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.442+0000 7faaa2ffd700 1 -- 172.21.15.62:0/1171725258 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaac00bc10 con 0x7faab0105820 2024-05-08T22:30:21.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.442+0000 7faab7f73700 1 -- 172.21.15.62:0/1171725258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab00fcdc0 con 0x7faab0105820 2024-05-08T22:30:21.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.442+0000 7faaa2ffd700 1 -- 172.21.15.62:0/1171725258 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faaac0038f0 con 0x7faab0105820 2024-05-08T22:30:21.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.442+0000 7faab7f73700 1 -- 172.21.15.62:0/1171725258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab00fd1e0 con 0x7faab0105820 2024-05-08T22:30:21.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.443+0000 7faaa2ffd700 1 -- 172.21.15.62:0/1171725258 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7faaac003b60 con 0x7faab0105820 2024-05-08T22:30:21.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.443+0000 7faab7f73700 1 -- 172.21.15.62:0/1171725258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab00ff330 con 0x7faab0105820 2024-05-08T22:30:21.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.443+0000 7faaa2ffd700 1 --2- 172.21.15.62:0/1171725258 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faa9c0387a0 0x7faa9c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:21.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.443+0000 7faaa2ffd700 1 -- 172.21.15.62:0/1171725258 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faaac04c430 con 0x7faab0105820 2024-05-08T22:30:21.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.443+0000 7faab550e700 1 --2- 172.21.15.62:0/1171725258 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faa9c0387a0 0x7faa9c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:21.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.444+0000 7faab550e700 1 --2- 172.21.15.62:0/1171725258 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faa9c0387a0 0x7faa9c03ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faaa4006fd0 tx=0x7faaa4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:21.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.448+0000 7faaa2ffd700 1 -- 172.21.15.62:0/1171725258 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faaac029540 con 0x7faab0105820 2024-05-08T22:30:21.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.655+0000 7faab7f73700 1 -- 172.21.15.62:0/1171725258 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faab00fd420 con 0x7faa9c0387a0 2024-05-08T22:30:21.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.656+0000 7faaa2ffd700 1 -- 172.21.15.62:0/1171725258 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7faab00fd420 con 0x7faa9c0387a0 2024-05-08T22:30:21.656 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:21.656 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}] 2024-05-08T22:30:21.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.658+0000 7faab7f73700 1 -- 172.21.15.62:0/1171725258 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faa9c0387a0 msgr2=0x7faa9c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:21.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.658+0000 7faab7f73700 1 --2- 172.21.15.62:0/1171725258 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faa9c0387a0 0x7faa9c03ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faaa4006fd0 tx=0x7faaa4006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:21.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.658+0000 7faab7f73700 1 -- 172.21.15.62:0/1171725258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab0105820 msgr2=0x7faab00fc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:21.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.658+0000 7faab7f73700 1 --2- 172.21.15.62:0/1171725258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab0105820 0x7faab00fc880 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faaac006010 tx=0x7faaac004ff0 comp rx=0 tx=0).stop 2024-05-08T22:30:21.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.658+0000 7faab7f73700 1 -- 172.21.15.62:0/1171725258 shutdown_connections 2024-05-08T22:30:21.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.658+0000 7faab7f73700 1 --2- 172.21.15.62:0/1171725258 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faa9c0387a0 0x7faa9c03ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:21.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.658+0000 7faab7f73700 1 --2- 172.21.15.62:0/1171725258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab0105820 0x7faab00fc880 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:21.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.658+0000 7faab7f73700 1 -- 172.21.15.62:0/1171725258 >> 172.21.15.62:0/1171725258 conn(0x7faab0075350 msgr2=0x7faab00680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:21.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.658+0000 7faab7f73700 1 -- 172.21.15.62:0/1171725258 shutdown_connections 2024-05-08T22:30:21.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:21.658+0000 7faab7f73700 1 -- 172.21.15.62:0/1171725258 wait complete. 2024-05-08T22:30:22.024 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi079 2024-05-08T22:30:22.024 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-08T22:30:22.024 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/etc/ceph/ceph.conf 2024-05-08T22:30:22.045 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-08T22:30:22.046 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:22.105 INFO:tasks.cephadm:Adding host smithi079 to orchestrator... 2024-05-08T22:30:22.106 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch host add smithi079 2024-05-08T22:30:22.329 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:30:22.501 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:22 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:22.501 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:22 smithi062 ceph-mon[42571]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:30:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.358+0000 7fe50950a700 1 -- 172.21.15.62:0/1588499861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe504105910 msgr2=0x7fe504105cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.358+0000 7fe50950a700 1 --2- 172.21.15.62:0/1588499861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe504105910 0x7fe504105cf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009910 tx=0x7fe4f4009c20 comp rx=0 tx=0).stop 2024-05-08T22:30:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.358+0000 7fe50950a700 1 -- 172.21.15.62:0/1588499861 shutdown_connections 2024-05-08T22:30:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.358+0000 7fe50950a700 1 --2- 172.21.15.62:0/1588499861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe504105910 0x7fe504105cf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.358+0000 7fe50950a700 1 -- 172.21.15.62:0/1588499861 >> 172.21.15.62:0/1588499861 conn(0x7fe5040faa50 msgr2=0x7fe5040fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.358+0000 7fe50950a700 1 -- 172.21.15.62:0/1588499861 shutdown_connections 2024-05-08T22:30:23.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.358+0000 7fe50950a700 1 -- 172.21.15.62:0/1588499861 wait complete. 2024-05-08T22:30:23.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.358+0000 7fe50950a700 1 Processor -- start 2024-05-08T22:30:23.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe50950a700 1 -- start start 2024-05-08T22:30:23.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe50950a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe504105910 0x7fe5040fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:23.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe50950a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe504101dd0 con 0x7fe504105910 2024-05-08T22:30:23.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe502ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe504105910 0x7fe5040fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:23.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe502ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe504105910 0x7fe5040fff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52706/0 (socket says 172.21.15.62:52706) 2024-05-08T22:30:23.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe502ffd700 1 -- 172.21.15.62:0/1024827723 learned_addr learned my addr 172.21.15.62:0/1024827723 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:23.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe502ffd700 1 -- 172.21.15.62:0/1024827723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f40095f0 con 0x7fe504105910 2024-05-08T22:30:23.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe502ffd700 1 --2- 172.21.15.62:0/1024827723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe504105910 0x7fe5040fff10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4004f40 tx=0x7fe4f4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:23.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe4ebfff700 1 -- 172.21.15.62:0/1024827723 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4f401d070 con 0x7fe504105910 2024-05-08T22:30:23.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe4ebfff700 1 -- 172.21.15.62:0/1024827723 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4f400bc10 con 0x7fe504105910 2024-05-08T22:30:23.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe50950a700 1 -- 172.21.15.62:0/1024827723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe504100450 con 0x7fe504105910 2024-05-08T22:30:23.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.359+0000 7fe4ebfff700 1 -- 172.21.15.62:0/1024827723 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4f40037d0 con 0x7fe504105910 2024-05-08T22:30:23.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.360+0000 7fe50950a700 1 -- 172.21.15.62:0/1024827723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe504100870 con 0x7fe504105910 2024-05-08T22:30:23.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.360+0000 7fe50950a700 1 -- 172.21.15.62:0/1024827723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5040625d0 con 0x7fe504105910 2024-05-08T22:30:23.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.361+0000 7fe4ebfff700 1 -- 172.21.15.62:0/1024827723 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fe4f400b6f0 con 0x7fe504105910 2024-05-08T22:30:23.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.361+0000 7fe4ebfff700 1 --2- 172.21.15.62:0/1024827723 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4ec0387a0 0x7fe4ec03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:23.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.361+0000 7fe4ebfff700 1 -- 172.21.15.62:0/1024827723 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe4f404c9a0 con 0x7fe504105910 2024-05-08T22:30:23.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.361+0000 7fe5027fc700 1 --2- 172.21.15.62:0/1024827723 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4ec0387a0 0x7fe4ec03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:23.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.361+0000 7fe5027fc700 1 --2- 172.21.15.62:0/1024827723 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4ec0387a0 0x7fe4ec03ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8006fd0 tx=0x7fe4f8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:23.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.365+0000 7fe4ebfff700 1 -- 172.21.15.62:0/1024827723 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe4f4004440 con 0x7fe504105910 2024-05-08T22:30:23.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:23.575+0000 7fe50950a700 1 -- 172.21.15.62:0/1024827723 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi079", "target": ["mon-mgr", ""]}) v1 -- 0x7fe504100ab0 con 0x7fe4ec0387a0 2024-05-08T22:30:24.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:24.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:24.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:24.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:24.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:30:24.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:24.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:24.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:24 smithi062 ceph-mon[42571]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi079", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:24.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:24 smithi062 ceph-mon[42571]: Updating smithi066:/etc/ceph/ceph.conf 2024-05-08T22:30:24.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:24 smithi062 ceph-mon[42571]: Updating smithi066:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:25.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:25.832+0000 7fe4ebfff700 1 -- 172.21.15.62:0/1024827723 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fe504100ab0 con 0x7fe4ec0387a0 2024-05-08T22:30:25.833 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi079' with addr '172.21.15.79' 2024-05-08T22:30:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:25.834+0000 7fe50950a700 1 -- 172.21.15.62:0/1024827723 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4ec0387a0 msgr2=0x7fe4ec03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:25.834+0000 7fe50950a700 1 --2- 172.21.15.62:0/1024827723 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4ec0387a0 0x7fe4ec03ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8006fd0 tx=0x7fe4f8006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:25.834+0000 7fe50950a700 1 -- 172.21.15.62:0/1024827723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe504105910 msgr2=0x7fe5040fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:25.834+0000 7fe50950a700 1 --2- 172.21.15.62:0/1024827723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe504105910 0x7fe5040fff10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4004f40 tx=0x7fe4f4005020 comp rx=0 tx=0).stop 2024-05-08T22:30:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:25.835+0000 7fe50950a700 1 -- 172.21.15.62:0/1024827723 shutdown_connections 2024-05-08T22:30:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:25.835+0000 7fe50950a700 1 --2- 172.21.15.62:0/1024827723 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4ec0387a0 0x7fe4ec03ac60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:25.835+0000 7fe50950a700 1 --2- 172.21.15.62:0/1024827723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe504105910 0x7fe5040fff10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:25.835+0000 7fe50950a700 1 -- 172.21.15.62:0/1024827723 >> 172.21.15.62:0/1024827723 conn(0x7fe5040faa50 msgr2=0x7fe5041084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:25.835+0000 7fe50950a700 1 -- 172.21.15.62:0/1024827723 shutdown_connections 2024-05-08T22:30:25.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:25.835+0000 7fe50950a700 1 -- 172.21.15.62:0/1024827723 wait complete. 2024-05-08T22:30:25.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:25 smithi062 ceph-mon[42571]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:25.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:25 smithi062 ceph-mon[42571]: Deploying cephadm binary to smithi079 2024-05-08T22:30:25.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:25 smithi062 ceph-mon[42571]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:25.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:25 smithi062 ceph-mon[42571]: Updating smithi066:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:30:25.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:25 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:25.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:25 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:25.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:25 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:26.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T22:30:26.424 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:30:27.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:26 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:27.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:26 smithi062 ceph-mon[42571]: Added host smithi079 2024-05-08T22:30:27.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:26 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:27.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:26 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:27.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:26 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:27.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.381+0000 7f48c9814700 1 -- 172.21.15.62:0/799263987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4105820 msgr2=0x7f48c4105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:27.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.381+0000 7f48c9814700 1 --2- 172.21.15.62:0/799263987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4105820 0x7f48c4105c00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009910 tx=0x7f48b4009c20 comp rx=0 tx=0).stop 2024-05-08T22:30:27.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.381+0000 7f48c9814700 1 -- 172.21.15.62:0/799263987 shutdown_connections 2024-05-08T22:30:27.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.381+0000 7f48c9814700 1 --2- 172.21.15.62:0/799263987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4105820 0x7f48c4105c00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:27.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.381+0000 7f48c9814700 1 -- 172.21.15.62:0/799263987 >> 172.21.15.62:0/799263987 conn(0x7f48c4075350 msgr2=0x7f48c4075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:27.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.381+0000 7f48c9814700 1 -- 172.21.15.62:0/799263987 shutdown_connections 2024-05-08T22:30:27.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.381+0000 7f48c9814700 1 -- 172.21.15.62:0/799263987 wait complete. 2024-05-08T22:30:27.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.382+0000 7f48c9814700 1 Processor -- start 2024-05-08T22:30:27.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.382+0000 7f48c9814700 1 -- start start 2024-05-08T22:30:27.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.382+0000 7f48c9814700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4105820 0x7f48c41961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:27.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.382+0000 7f48c9814700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c41966f0 con 0x7f48c4105820 2024-05-08T22:30:27.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.382+0000 7f48c2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4105820 0x7f48c41961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:27.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.382+0000 7f48c2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4105820 0x7f48c41961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40528/0 (socket says 172.21.15.62:40528) 2024-05-08T22:30:27.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.382+0000 7f48c2ffd700 1 -- 172.21.15.62:0/3946102874 learned_addr learned my addr 172.21.15.62:0/3946102874 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:27.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.382+0000 7f48c2ffd700 1 -- 172.21.15.62:0/3946102874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b40095f0 con 0x7f48c4105820 2024-05-08T22:30:27.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.383+0000 7f48c2ffd700 1 --2- 172.21.15.62:0/3946102874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4105820 0x7f48c41961b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48b4004f40 tx=0x7f48b4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:27.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.383+0000 7f48c8812700 1 -- 172.21.15.62:0/3946102874 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48b401d070 con 0x7f48c4105820 2024-05-08T22:30:27.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.383+0000 7f48c9814700 1 -- 172.21.15.62:0/3946102874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c41968f0 con 0x7f48c4105820 2024-05-08T22:30:27.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.383+0000 7f48c8812700 1 -- 172.21.15.62:0/3946102874 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48b4004370 con 0x7f48c4105820 2024-05-08T22:30:27.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.383+0000 7f48c8812700 1 -- 172.21.15.62:0/3946102874 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48b4003970 con 0x7f48c4105820 2024-05-08T22:30:27.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.383+0000 7f48c9814700 1 -- 172.21.15.62:0/3946102874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c4196d10 con 0x7f48c4105820 2024-05-08T22:30:27.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.384+0000 7f48c8812700 1 -- 172.21.15.62:0/3946102874 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f48b4003c30 con 0x7f48c4105820 2024-05-08T22:30:27.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.384+0000 7f48c9814700 1 -- 172.21.15.62:0/3946102874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c40ff330 con 0x7f48c4105820 2024-05-08T22:30:27.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.384+0000 7f48c8812700 1 --2- 172.21.15.62:0/3946102874 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48ac0387a0 0x7f48ac03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:27.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.384+0000 7f48c8812700 1 -- 172.21.15.62:0/3946102874 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f48b404c5f0 con 0x7f48c4105820 2024-05-08T22:30:27.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.384+0000 7f48c27fc700 1 --2- 172.21.15.62:0/3946102874 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48ac0387a0 0x7f48ac03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:27.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.384+0000 7f48c27fc700 1 --2- 172.21.15.62:0/3946102874 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48ac0387a0 0x7f48ac03ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f48b8006fd0 tx=0x7f48b8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.389+0000 7f48c8812700 1 -- 172.21.15.62:0/3946102874 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f48b4029950 con 0x7f48c4105820 2024-05-08T22:30:27.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.598+0000 7f48c9814700 1 -- 172.21.15.62:0/3946102874 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48c4109b30 con 0x7f48ac0387a0 2024-05-08T22:30:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.599+0000 7f48c8812700 1 -- 172.21.15.62:0/3946102874 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+238 (secure 0 0 0) 0x7f48c4109b30 con 0x7f48ac0387a0 2024-05-08T22:30:27.599 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:30:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.79", "hostname": "smithi079", "labels": [], "status": ""}] 2024-05-08T22:30:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.601+0000 7f48c9814700 1 -- 172.21.15.62:0/3946102874 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48ac0387a0 msgr2=0x7f48ac03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.601+0000 7f48c9814700 1 --2- 172.21.15.62:0/3946102874 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48ac0387a0 0x7f48ac03ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f48b8006fd0 tx=0x7f48b8006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.601+0000 7f48c9814700 1 -- 172.21.15.62:0/3946102874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4105820 msgr2=0x7f48c41961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.601+0000 7f48c9814700 1 --2- 172.21.15.62:0/3946102874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4105820 0x7f48c41961b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48b4004f40 tx=0x7f48b4005020 comp rx=0 tx=0).stop 2024-05-08T22:30:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.601+0000 7f48c9814700 1 -- 172.21.15.62:0/3946102874 shutdown_connections 2024-05-08T22:30:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.601+0000 7f48c9814700 1 --2- 172.21.15.62:0/3946102874 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48ac0387a0 0x7f48ac03ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.601+0000 7f48c9814700 1 --2- 172.21.15.62:0/3946102874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4105820 0x7f48c41961b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:27.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.601+0000 7f48c9814700 1 -- 172.21.15.62:0/3946102874 >> 172.21.15.62:0/3946102874 conn(0x7f48c4075350 msgr2=0x7f48c40680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:27.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.601+0000 7f48c9814700 1 -- 172.21.15.62:0/3946102874 shutdown_connections 2024-05-08T22:30:27.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:27.601+0000 7f48c9814700 1 -- 172.21.15.62:0/3946102874 wait complete. 2024-05-08T22:30:27.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:27 smithi062 ceph-mon[42571]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:27.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:27.956 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-08T22:30:27.956 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-08T22:30:28.179 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:30:29.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.115+0000 7f2eb12ad700 1 -- 172.21.15.62:0/965951529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eac1068c0 msgr2=0x7f2eac106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:29.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.115+0000 7f2eb12ad700 1 --2- 172.21.15.62:0/965951529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eac1068c0 0x7f2eac106ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009910 tx=0x7f2e9c009c20 comp rx=0 tx=0).stop 2024-05-08T22:30:29.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.115+0000 7f2eb12ad700 1 -- 172.21.15.62:0/965951529 shutdown_connections 2024-05-08T22:30:29.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.115+0000 7f2eb12ad700 1 --2- 172.21.15.62:0/965951529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eac1068c0 0x7f2eac106ca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.115+0000 7f2eb12ad700 1 -- 172.21.15.62:0/965951529 >> 172.21.15.62:0/965951529 conn(0x7f2eac075350 msgr2=0x7f2eac075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.115+0000 7f2eb12ad700 1 -- 172.21.15.62:0/965951529 shutdown_connections 2024-05-08T22:30:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.115+0000 7f2eb12ad700 1 -- 172.21.15.62:0/965951529 wait complete. 2024-05-08T22:30:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.116+0000 7f2eb12ad700 1 Processor -- start 2024-05-08T22:30:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.116+0000 7f2eb12ad700 1 -- start start 2024-05-08T22:30:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.116+0000 7f2eb12ad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eac1068c0 0x7f2eac0ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.116+0000 7f2eb12ad700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac101760 con 0x7f2eac1068c0 2024-05-08T22:30:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.117+0000 7f2eaaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eac1068c0 0x7f2eac0ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.117+0000 7f2eaaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eac1068c0 0x7f2eac0ff8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40544/0 (socket says 172.21.15.62:40544) 2024-05-08T22:30:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.117+0000 7f2eaaffd700 1 -- 172.21.15.62:0/3161011735 learned_addr learned my addr 172.21.15.62:0/3161011735 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.117+0000 7f2eaaffd700 1 -- 172.21.15.62:0/3161011735 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0095f0 con 0x7f2eac1068c0 2024-05-08T22:30:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.117+0000 7f2eaaffd700 1 --2- 172.21.15.62:0/3161011735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eac1068c0 0x7f2eac0ff8a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c004f40 tx=0x7f2e9c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.117+0000 7f2ea3fff700 1 -- 172.21.15.62:0/3161011735 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c01d070 con 0x7f2eac1068c0 2024-05-08T22:30:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.117+0000 7f2eb12ad700 1 -- 172.21.15.62:0/3161011735 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac0ffde0 con 0x7f2eac1068c0 2024-05-08T22:30:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.117+0000 7f2ea3fff700 1 -- 172.21.15.62:0/3161011735 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e9c00bbb0 con 0x7f2eac1068c0 2024-05-08T22:30:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.117+0000 7f2ea3fff700 1 -- 172.21.15.62:0/3161011735 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c003810 con 0x7f2eac1068c0 2024-05-08T22:30:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.117+0000 7f2eb12ad700 1 -- 172.21.15.62:0/3161011735 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac100260 con 0x7f2eac1068c0 2024-05-08T22:30:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.118+0000 7f2ea3fff700 1 -- 172.21.15.62:0/3161011735 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f2e9c003a90 con 0x7f2eac1068c0 2024-05-08T22:30:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.118+0000 7f2ea3fff700 1 --2- 172.21.15.62:0/3161011735 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e980387a0 0x7f2e9803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.118+0000 7f2ea3fff700 1 -- 172.21.15.62:0/3161011735 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2e9c04c4b0 con 0x7f2eac1068c0 2024-05-08T22:30:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.118+0000 7f2eaa7fc700 1 --2- 172.21.15.62:0/3161011735 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e980387a0 0x7f2e9803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.119+0000 7f2eaa7fc700 1 --2- 172.21.15.62:0/3161011735 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e980387a0 0x7f2e9803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e94006fd0 tx=0x7f2e94006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.119+0000 7f2eb12ad700 1 -- 172.21.15.62:0/3161011735 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eac102000 con 0x7f2eac1068c0 2024-05-08T22:30:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.123+0000 7f2ea3fff700 1 -- 172.21.15.62:0/3161011735 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2e9c026020 con 0x7f2eac1068c0 2024-05-08T22:30:29.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:28 smithi062 ceph-mon[42571]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:30:29.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.332+0000 7f2eb12ad700 1 -- 172.21.15.62:0/3161011735 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f2eac100e20 con 0x7f2eac1068c0 2024-05-08T22:30:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.839+0000 7f2ea3fff700 1 -- 172.21.15.62:0/3161011735 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f2e9c003e30 con 0x7f2eac1068c0 2024-05-08T22:30:29.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.841+0000 7f2eb12ad700 1 -- 172.21.15.62:0/3161011735 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e980387a0 msgr2=0x7f2e9803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:29.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.841+0000 7f2eb12ad700 1 --2- 172.21.15.62:0/3161011735 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e980387a0 0x7f2e9803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e94006fd0 tx=0x7f2e94006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:29.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.841+0000 7f2eb12ad700 1 -- 172.21.15.62:0/3161011735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eac1068c0 msgr2=0x7f2eac0ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:29.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.841+0000 7f2eb12ad700 1 --2- 172.21.15.62:0/3161011735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eac1068c0 0x7f2eac0ff8a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c004f40 tx=0x7f2e9c005020 comp rx=0 tx=0).stop 2024-05-08T22:30:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.841+0000 7f2eb12ad700 1 -- 172.21.15.62:0/3161011735 shutdown_connections 2024-05-08T22:30:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.841+0000 7f2eb12ad700 1 --2- 172.21.15.62:0/3161011735 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e980387a0 0x7f2e9803ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.841+0000 7f2eb12ad700 1 --2- 172.21.15.62:0/3161011735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2eac1068c0 0x7f2eac0ff8a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.841+0000 7f2eb12ad700 1 -- 172.21.15.62:0/3161011735 >> 172.21.15.62:0/3161011735 conn(0x7f2eac075350 msgr2=0x7f2eac0fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.842+0000 7f2eb12ad700 1 -- 172.21.15.62:0/3161011735 shutdown_connections 2024-05-08T22:30:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:29.842+0000 7f2eb12ad700 1 -- 172.21.15.62:0/3161011735 wait complete. 2024-05-08T22:30:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2024-05-08T22:30:30.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:29 smithi062 ceph-mon[42571]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:30.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:29 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3161011735' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T22:30:30.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:30.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:30.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:30.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:30.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:30:30.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:30.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:30.223 INFO:tasks.cephadm:Adding mon.a on smithi062 2024-05-08T22:30:30.224 INFO:tasks.cephadm:Adding mon.b on smithi066 2024-05-08T22:30:30.224 INFO:tasks.cephadm:Adding mon.c on smithi079 2024-05-08T22:30:30.225 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi062:172.21.15.62=a;smithi066:172.21.15.66=b;smithi079:172.21.15.79=c' 2024-05-08T22:30:30.472 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:31.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:30 smithi062 ceph-mon[42571]: Updating smithi079:/etc/ceph/ceph.conf 2024-05-08T22:30:31.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:30 smithi062 ceph-mon[42571]: Updating smithi079:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:31.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:30 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3161011735' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T22:30:31.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:30 smithi062 ceph-mon[42571]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:30:31.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:30 smithi062 ceph-mon[42571]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:31.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:30 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:31.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:30 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:31.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:30 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:31.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.421+0000 7f8b5e587700 1 -- 172.21.15.79:0/1885936738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b580ff020 msgr2=0x7f8b580ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:31.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.421+0000 7f8b5e587700 1 --2- 172.21.15.79:0/1885936738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b580ff020 0x7f8b580ff400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009910 tx=0x7f8b40009c20 comp rx=0 tx=0).stop 2024-05-08T22:30:31.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.421+0000 7f8b5e587700 1 -- 172.21.15.79:0/1885936738 shutdown_connections 2024-05-08T22:30:31.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.421+0000 7f8b5e587700 1 --2- 172.21.15.79:0/1885936738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b580ff020 0x7f8b580ff400 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:31.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.421+0000 7f8b5e587700 1 -- 172.21.15.79:0/1885936738 >> 172.21.15.79:0/1885936738 conn(0x7f8b580faa00 msgr2=0x7f8b580fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:31.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.422+0000 7f8b5e587700 1 -- 172.21.15.79:0/1885936738 shutdown_connections 2024-05-08T22:30:31.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.422+0000 7f8b5e587700 1 -- 172.21.15.79:0/1885936738 wait complete. 2024-05-08T22:30:31.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.422+0000 7f8b5e587700 1 Processor -- start 2024-05-08T22:30:31.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.422+0000 7f8b5e587700 1 -- start start 2024-05-08T22:30:31.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.422+0000 7f8b5e587700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b580ff020 0x7f8b581961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:31.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.422+0000 7f8b5e587700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581966f0 con 0x7f8b580ff020 2024-05-08T22:30:31.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.423+0000 7f8b57fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b580ff020 0x7f8b581961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:31.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.423+0000 7f8b57fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b580ff020 0x7f8b581961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.79:48748/0 (socket says 172.21.15.79:48748) 2024-05-08T22:30:31.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.423+0000 7f8b57fff700 1 -- 172.21.15.79:0/476441010 learned_addr learned my addr 172.21.15.79:0/476441010 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.423+0000 7f8b57fff700 1 -- 172.21.15.79:0/476441010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b400095f0 con 0x7f8b580ff020 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.423+0000 7f8b57fff700 1 --2- 172.21.15.79:0/476441010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b580ff020 0x7f8b581961b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8b40006010 tx=0x7f8b40004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.424+0000 7f8b557fa700 1 -- 172.21.15.79:0/476441010 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b4001d070 con 0x7f8b580ff020 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.424+0000 7f8b5e587700 1 -- 172.21.15.79:0/476441010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b581968f0 con 0x7f8b580ff020 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.424+0000 7f8b557fa700 1 -- 172.21.15.79:0/476441010 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b4000bc10 con 0x7f8b580ff020 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.424+0000 7f8b557fa700 1 -- 172.21.15.79:0/476441010 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b400038a0 con 0x7f8b580ff020 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.424+0000 7f8b5e587700 1 -- 172.21.15.79:0/476441010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b58196d10 con 0x7f8b580ff020 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.425+0000 7f8b5e587700 1 -- 172.21.15.79:0/476441010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b58108a90 con 0x7f8b580ff020 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.425+0000 7f8b557fa700 1 -- 172.21.15.79:0/476441010 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f8b4000b6f0 con 0x7f8b580ff020 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.425+0000 7f8b557fa700 1 --2- 172.21.15.79:0/476441010 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8b440387f0 0x7f8b4403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.425+0000 7f8b557fa700 1 -- 172.21.15.79:0/476441010 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8b4004c3b0 con 0x7f8b580ff020 2024-05-08T22:30:31.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.425+0000 7f8b577fe700 1 --2- 172.21.15.79:0/476441010 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8b440387f0 0x7f8b4403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:31.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.425+0000 7f8b577fe700 1 --2- 172.21.15.79:0/476441010 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8b440387f0 0x7f8b4403acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b48006fd0 tx=0x7f8b48006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:31.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.430+0000 7f8b557fa700 1 -- 172.21.15.79:0/476441010 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8b40003a00 con 0x7f8b580ff020 2024-05-08T22:30:31.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.631+0000 7f8b5e587700 1 -- 172.21.15.79:0/476441010 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi066:172.21.15.66=b;smithi079:172.21.15.79=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b5810ac00 con 0x7f8b440387f0 2024-05-08T22:30:31.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.638+0000 7f8b557fa700 1 -- 172.21.15.79:0/476441010 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8b5810ac00 con 0x7f8b440387f0 2024-05-08T22:30:31.637 INFO:teuthology.orchestra.run.smithi079.stdout:Scheduled mon update... 2024-05-08T22:30:31.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.639+0000 7f8b5e587700 1 -- 172.21.15.79:0/476441010 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8b440387f0 msgr2=0x7f8b4403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:31.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.639+0000 7f8b5e587700 1 --2- 172.21.15.79:0/476441010 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8b440387f0 0x7f8b4403acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b48006fd0 tx=0x7f8b48006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:31.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.640+0000 7f8b5e587700 1 -- 172.21.15.79:0/476441010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b580ff020 msgr2=0x7f8b581961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:31.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.640+0000 7f8b5e587700 1 --2- 172.21.15.79:0/476441010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b580ff020 0x7f8b581961b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8b40006010 tx=0x7f8b40004ff0 comp rx=0 tx=0).stop 2024-05-08T22:30:31.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.640+0000 7f8b5e587700 1 -- 172.21.15.79:0/476441010 shutdown_connections 2024-05-08T22:30:31.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.640+0000 7f8b5e587700 1 --2- 172.21.15.79:0/476441010 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8b440387f0 0x7f8b4403acb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:31.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.640+0000 7f8b5e587700 1 --2- 172.21.15.79:0/476441010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b580ff020 0x7f8b581961b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:31.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.640+0000 7f8b5e587700 1 -- 172.21.15.79:0/476441010 >> 172.21.15.79:0/476441010 conn(0x7f8b580faa00 msgr2=0x7f8b581024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:31.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.640+0000 7f8b5e587700 1 -- 172.21.15.79:0/476441010 shutdown_connections 2024-05-08T22:30:31.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:31.640+0000 7f8b5e587700 1 -- 172.21.15.79:0/476441010 wait complete. 2024-05-08T22:30:32.026 DEBUG:teuthology.orchestra.run.smithi066:mon.b> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.b.service 2024-05-08T22:30:32.029 DEBUG:teuthology.orchestra.run.smithi079:mon.c> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.c.service 2024-05-08T22:30:32.031 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T22:30:32.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T22:30:32.061 INFO:journalctl@ceph.mon.c.smithi079.stdout:-- Logs begin at Wed 2024-05-08 22:12:11 UTC. -- 2024-05-08T22:30:32.066 INFO:journalctl@ceph.mon.b.smithi066.stdout:-- Logs begin at Wed 2024-05-08 22:13:17 UTC. -- 2024-05-08T22:30:32.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:31 smithi062 ceph-mon[42571]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:32.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:31 smithi062 ceph-mon[42571]: Updating smithi079:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:30:32.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:32.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:32.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:32.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:32.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:32.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:32.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:32.626 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:33.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:32 smithi062 ceph-mon[42571]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi066:172.21.15.66=b;smithi079:172.21.15.79=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:33.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:32 smithi062 ceph-mon[42571]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi066:172.21.15.66=b;smithi079:172.21.15.79=c;count:3 2024-05-08T22:30:33.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:32 smithi062 ceph-mon[42571]: Deploying daemon mon.c on smithi079 2024-05-08T22:30:34.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.061+0000 7f4d743ff700 1 -- 172.21.15.79:0/3325655735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c101b30 msgr2=0x7f4d6c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:34.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.061+0000 7f4d743ff700 1 --2- 172.21.15.79:0/3325655735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c101b30 0x7f4d6c101f10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400b0a0 tx=0x7f4d6400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:30:34.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.061+0000 7f4d743ff700 1 -- 172.21.15.79:0/3325655735 shutdown_connections 2024-05-08T22:30:34.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.061+0000 7f4d743ff700 1 --2- 172.21.15.79:0/3325655735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c101b30 0x7f4d6c101f10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:34.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.061+0000 7f4d743ff700 1 -- 172.21.15.79:0/3325655735 >> 172.21.15.79:0/3325655735 conn(0x7f4d6c0fb2e0 msgr2=0x7f4d6c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:34.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.061+0000 7f4d743ff700 1 -- 172.21.15.79:0/3325655735 shutdown_connections 2024-05-08T22:30:34.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.062+0000 7f4d743ff700 1 -- 172.21.15.79:0/3325655735 wait complete. 2024-05-08T22:30:34.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.062+0000 7f4d743ff700 1 Processor -- start 2024-05-08T22:30:34.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.062+0000 7f4d743ff700 1 -- start start 2024-05-08T22:30:34.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.063+0000 7f4d743ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c101b30 0x7f4d6c1961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:34.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.063+0000 7f4d743ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c196730 con 0x7f4d6c101b30 2024-05-08T22:30:34.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.063+0000 7f4d7219b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c101b30 0x7f4d6c1961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:34.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.063+0000 7f4d7219b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c101b30 0x7f4d6c1961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.79:48776/0 (socket says 172.21.15.79:48776) 2024-05-08T22:30:34.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.063+0000 7f4d7219b700 1 -- 172.21.15.79:0/1761699399 learned_addr learned my addr 172.21.15.79:0/1761699399 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-08T22:30:34.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.063+0000 7f4d7219b700 1 -- 172.21.15.79:0/1761699399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d64009d00 con 0x7f4d6c101b30 2024-05-08T22:30:34.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.063+0000 7f4d7219b700 1 --2- 172.21.15.79:0/1761699399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c101b30 0x7f4d6c1961f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4d64015040 tx=0x7f4d640096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:34.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.063+0000 7f4d6b7fe700 1 -- 172.21.15.79:0/1761699399 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d6400e040 con 0x7f4d6c101b30 2024-05-08T22:30:34.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.063+0000 7f4d6b7fe700 1 -- 172.21.15.79:0/1761699399 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d64004040 con 0x7f4d6c101b30 2024-05-08T22:30:34.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.063+0000 7f4d743ff700 1 -- 172.21.15.79:0/1761699399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c196930 con 0x7f4d6c101b30 2024-05-08T22:30:34.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.063+0000 7f4d6b7fe700 1 -- 172.21.15.79:0/1761699399 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d6401d610 con 0x7f4d6c101b30 2024-05-08T22:30:34.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.064+0000 7f4d743ff700 1 -- 172.21.15.79:0/1761699399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c196d50 con 0x7f4d6c101b30 2024-05-08T22:30:34.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.064+0000 7f4d743ff700 1 -- 172.21.15.79:0/1761699399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6c0691c0 con 0x7f4d6c101b30 2024-05-08T22:30:34.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.064+0000 7f4d6b7fe700 1 -- 172.21.15.79:0/1761699399 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f4d64007b10 con 0x7f4d6c101b30 2024-05-08T22:30:34.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.065+0000 7f4d6b7fe700 1 --2- 172.21.15.79:0/1761699399 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4d540387f0 0x7f4d5403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:34.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.065+0000 7f4d6b7fe700 1 -- 172.21.15.79:0/1761699399 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4d6404c8b0 con 0x7f4d6c101b30 2024-05-08T22:30:34.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.065+0000 7f4d7199a700 1 --2- 172.21.15.79:0/1761699399 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4d540387f0 0x7f4d5403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:34.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.065+0000 7f4d7199a700 1 --2- 172.21.15.79:0/1761699399 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4d540387f0 0x7f4d5403acb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4d60006fd0 tx=0x7f4d60006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:34.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.070+0000 7f4d6b7fe700 1 -- 172.21.15.79:0/1761699399 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4d6401c5c0 con 0x7f4d6c101b30 2024-05-08T22:30:34.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:33 smithi062 ceph-mon[42571]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:34.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.386+0000 7f4d743ff700 1 -- 172.21.15.79:0/1761699399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4d6c0625d0 con 0x7f4d6c101b30 2024-05-08T22:30:34.388 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:30:34.388 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":1,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","modified":"2024-05-08T22:29:13.064164Z","created":"2024-05-08T22:29:13.064164Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-08T22:30:34.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.387+0000 7f4d6b7fe700 1 -- 172.21.15.79:0/1761699399 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f4d64017030 con 0x7f4d6c101b30 2024-05-08T22:30:34.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.389+0000 7f4d743ff700 1 -- 172.21.15.79:0/1761699399 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4d540387f0 msgr2=0x7f4d5403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:34.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.389+0000 7f4d743ff700 1 --2- 172.21.15.79:0/1761699399 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4d540387f0 0x7f4d5403acb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4d60006fd0 tx=0x7f4d60006e40 comp rx=0 tx=0).stop 2024-05-08T22:30:34.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.389+0000 7f4d743ff700 1 -- 172.21.15.79:0/1761699399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c101b30 msgr2=0x7f4d6c1961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.389+0000 7f4d743ff700 1 --2- 172.21.15.79:0/1761699399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c101b30 0x7f4d6c1961f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4d64015040 tx=0x7f4d640096e0 comp rx=0 tx=0).stop 2024-05-08T22:30:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.389+0000 7f4d743ff700 1 -- 172.21.15.79:0/1761699399 shutdown_connections 2024-05-08T22:30:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.389+0000 7f4d743ff700 1 --2- 172.21.15.79:0/1761699399 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4d540387f0 0x7f4d5403acb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.389+0000 7f4d743ff700 1 --2- 172.21.15.79:0/1761699399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c101b30 0x7f4d6c1961f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.389+0000 7f4d743ff700 1 -- 172.21.15.79:0/1761699399 >> 172.21.15.79:0/1761699399 conn(0x7f4d6c0fb2e0 msgr2=0x7f4d6c1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:34.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.389+0000 7f4d743ff700 1 -- 172.21.15.79:0/1761699399 shutdown_connections 2024-05-08T22:30:34.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:34.389+0000 7f4d743ff700 1 -- 172.21.15.79:0/1761699399 wait complete. 2024-05-08T22:30:34.392 INFO:teuthology.orchestra.run.smithi079.stderr:dumped monmap epoch 1 2024-05-08T22:30:35.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 systemd[1]: Started Ceph mon.c for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:30:35.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).mds e1 new map 2024-05-08T22:30:35.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-08T22:30:35.141 INFO:journalctl@ceph.mon.c.smithi079.stdout: e1 2024-05-08T22:30:35.141 INFO:journalctl@ceph.mon.c.smithi079.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T22:30:35.141 INFO:journalctl@ceph.mon.c.smithi079.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T22:30:35.141 INFO:journalctl@ceph.mon.c.smithi079.stdout: legacy client fscid: -1 2024-05-08T22:30:35.142 INFO:journalctl@ceph.mon.c.smithi079.stdout: 2024-05-08T22:30:35.142 INFO:journalctl@ceph.mon.c.smithi079.stdout: No filesystems configured 2024-05-08T22:30:35.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T22:30:35.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T22:30:35.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T22:30:35.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T22:30:35.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T22:30:35.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T22:30:35.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T22:30:35.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:30:35.144 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:30:35.144 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:30:35.144 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mkfs 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:30:35.144 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T22:30:35.144 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T22:30:35.144 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: monmap e1: 1 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:30:35.145 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: fsmap 2024-05-08T22:30:35.145 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T22:30:35.145 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e1: no daemons active 2024-05-08T22:30:35.145 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1366075249' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T22:30:35.145 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/402087504' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T22:30:35.145 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/402087504' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T22:30:35.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2608462915' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:35.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T22:30:35.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: monmap e1: 1 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:30:35.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: fsmap 2024-05-08T22:30:35.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T22:30:35.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e1: no daemons active 2024-05-08T22:30:35.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3243045878' entity='client.admin' 2024-05-08T22:30:35.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3948413767' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T22:30:35.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1767912247' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T22:30:35.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Activating manager daemon x 2024-05-08T22:30:35.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e2: x(active, starting, since 0.00328512s) 2024-05-08T22:30:35.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:30:35.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:30:35.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:30:35.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:30:35.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:30:35.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Manager daemon x is now available 2024-05-08T22:30:35.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:30:35.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:30:35.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' 2024-05-08T22:30:35.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' 2024-05-08T22:30:35.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14100 172.21.15.62:0/670182241' entity='mgr.x' 2024-05-08T22:30:35.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e3: x(active, since 1.00611s) 2024-05-08T22:30:35.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1820659283' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T22:30:35.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/4273590492' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T22:30:35.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2205659201' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T22:30:35.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2205659201' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T22:30:35.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e4: x(active, since 3s) 2024-05-08T22:30:35.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2986721068' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T22:30:35.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Active manager daemon x restarted 2024-05-08T22:30:35.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Activating manager daemon x 2024-05-08T22:30:35.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T22:30:35.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e5: x(active, starting, since 0.00345048s) 2024-05-08T22:30:35.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:30:35.152 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:30:35.152 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:30:35.152 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:30:35.152 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:30:35.152 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Manager daemon x is now available 2024-05-08T22:30:35.152 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.153 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.153 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Found migration_current of "None". Setting to last migration. 2024-05-08T22:30:35.153 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.153 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.153 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:30:35.153 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:30:35.154 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e6: x(active, since 1.00578s) 2024-05-08T22:30:35.154 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T22:30:35.154 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T22:30:35.154 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.154 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.154 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.155 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.155 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.155 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.155 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: [08/May/2024:22:29:45] ENGINE Bus STARTING 2024-05-08T22:30:35.155 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: [08/May/2024:22:29:45] ENGINE Serving on http://172.21.15.62:8765 2024-05-08T22:30:35.155 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: [08/May/2024:22:29:45] ENGINE Serving on https://172.21.15.62:7150 2024-05-08T22:30:35.155 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: [08/May/2024:22:29:45] ENGINE Bus STARTED 2024-05-08T22:30:35.156 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.156 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e7: x(active, since 2s) 2024-05-08T22:30:35.156 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.156 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Generating ssh key... 2024-05-08T22:30:35.156 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.156 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.157 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.157 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.157 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Deploying cephadm binary to smithi062 2024-05-08T22:30:35.157 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.157 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Added host smithi062 2024-05-08T22:30:35.157 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.158 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.158 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.158 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Saving service mon spec with placement count:5 2024-05-08T22:30:35.158 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.158 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.158 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.159 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.159 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Saving service mgr spec with placement count:2 2024-05-08T22:30:35.159 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.159 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14118 172.21.15.62:0/2486524198' entity='mgr.x' 2024-05-08T22:30:35.159 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3229075238' entity='client.admin' 2024-05-08T22:30:35.159 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1367795997' entity='client.admin' 2024-05-08T22:30:35.160 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1045871488' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T22:30:35.160 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1045871488' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T22:30:35.160 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e8: x(active, since 13s) 2024-05-08T22:30:35.160 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/4052717337' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T22:30:35.160 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Active manager daemon x restarted 2024-05-08T22:30:35.160 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Activating manager daemon x 2024-05-08T22:30:35.161 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T22:30:35.161 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e9: x(active, starting, since 0.0034978s) 2024-05-08T22:30:35.161 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:30:35.161 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T22:30:35.161 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T22:30:35.161 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T22:30:35.162 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T22:30:35.162 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Manager daemon x is now available 2024-05-08T22:30:35.162 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.162 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T22:30:35.162 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T22:30:35.162 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: [08/May/2024:22:30:04] ENGINE Bus STARTING 2024-05-08T22:30:35.163 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.163 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: [08/May/2024:22:30:04] ENGINE Serving on http://172.21.15.62:8765 2024-05-08T22:30:35.163 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e10: x(active, since 1.00634s) 2024-05-08T22:30:35.163 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T22:30:35.163 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T22:30:35.163 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: [08/May/2024:22:30:05] ENGINE Serving on https://172.21.15.62:7150 2024-05-08T22:30:35.164 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: [08/May/2024:22:30:05] ENGINE Bus STARTED 2024-05-08T22:30:35.164 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.164 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.164 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.164 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.164 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e11: x(active, since 2s) 2024-05-08T22:30:35.165 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.165 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.165 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1601957982' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T22:30:35.165 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mgrmap e12: x(active, since 6s) 2024-05-08T22:30:35.165 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3517234272' entity='client.admin' 2024-05-08T22:30:35.165 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:30:35.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.167 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2884077148' entity='client.admin' 2024-05-08T22:30:35.167 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.167 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.167 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.167 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.167 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.168 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:35.168 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:35.168 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi062:/etc/ceph/ceph.conf 2024-05-08T22:30:35.168 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi062:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:35.168 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:35.168 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi062:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:30:35.169 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.170 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.170 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.170 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi066", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.170 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Deploying cephadm binary to smithi066 2024-05-08T22:30:35.170 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.171 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Added host smithi066 2024-05-08T22:30:35.171 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.171 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.171 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.171 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.171 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:30:35.172 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.172 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.172 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.172 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.172 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:30:35.172 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:35.172 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:35.173 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi079", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.173 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi066:/etc/ceph/ceph.conf 2024-05-08T22:30:35.173 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi066:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:35.173 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:35.173 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Deploying cephadm binary to smithi079 2024-05-08T22:30:35.173 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:35.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi066:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:30:35.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Added host smithi079 2024-05-08T22:30:35.175 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.175 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.175 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.175 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:35.175 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.175 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:30:35.176 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:35.176 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3161011735' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T22:30:35.176 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.176 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.176 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.176 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.177 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:30:35.177 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:35.177 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:35.177 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi079:/etc/ceph/ceph.conf 2024-05-08T22:30:35.177 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi079:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:35.177 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3161011735' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T22:30:35.177 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:30:35.178 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:35.178 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.178 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.178 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.178 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:35.178 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Updating smithi079:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T22:30:35.179 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.179 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.179 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:35.179 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:35.179 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:35.179 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:35.180 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:35.180 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi066:172.21.15.66=b;smithi079:172.21.15.79=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:35.180 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi066:172.21.15.66=b;smithi079:172.21.15.79=c;count:3 2024-05-08T22:30:35.180 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: Deploying daemon mon.c on smithi079 2024-05-08T22:30:35.180 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:35.181 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.79:0/1761699399' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.181 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:34 smithi079 ceph-mon[46738]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T22:30:35.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:34 smithi062 ceph-mon[42571]: from='client.? 172.21.15.79:0/1761699399' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:30:35.817 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T22:30:35.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T22:30:36.046 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:30:40.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.085+0000 7f05f9272700 1 -- 172.21.15.79:0/1839427831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc0057d0 msgr2=0x7f05dc007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:40.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.085+0000 7f05f9272700 1 --2- 172.21.15.79:0/1839427831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc0057d0 0x7f05dc007c60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f05f40ff7f0 tx=0x7f05e800feb0 comp rx=0 tx=0).stop 2024-05-08T22:30:40.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.085+0000 7f05f9272700 1 -- 172.21.15.79:0/1839427831 shutdown_connections 2024-05-08T22:30:40.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.085+0000 7f05f9272700 1 --2- 172.21.15.79:0/1839427831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc0057d0 0x7f05dc007c60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:40.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.085+0000 7f05f9272700 1 -- 172.21.15.79:0/1839427831 >> 172.21.15.79:0/1839427831 conn(0x7f05f4076570 msgr2=0x7f05f40fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:40.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.085+0000 7f05f9272700 1 -- 172.21.15.79:0/1839427831 shutdown_connections 2024-05-08T22:30:40.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.085+0000 7f05f9272700 1 -- 172.21.15.79:0/1839427831 wait complete. 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.086+0000 7f05f9272700 1 Processor -- start 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f9272700 1 -- start start 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f9272700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc0057d0 0x7f05f4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f9272700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05f41989a0 0x7f05f419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f9272700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f4198f30 con 0x7f05dc0057d0 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f9272700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f41990a0 con 0x7f05f41989a0 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc0057d0 0x7f05f4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f27fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05f41989a0 0x7f05f419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc0057d0 0x7f05f4198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.79:48848/0 (socket says 172.21.15.79:48848) 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f2ffd700 1 -- 172.21.15.79:0/3501365071 learned_addr learned my addr 172.21.15.79:0/3501365071 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f27fc700 1 -- 172.21.15.79:0/3501365071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05f41989a0 msgr2=0x7f05f419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T22:30:40.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f27fc700 1 -- 172.21.15.79:0/3501365071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05f41989a0 msgr2=0x7f05f419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:30:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f27fc700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05f41989a0 0x7f05f419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:30:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f27fc700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05f41989a0 0x7f05f419cce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:30:40.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f2ffd700 1 -- 172.21.15.79:0/3501365071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05f41989a0 msgr2=0x7f05f419cce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:30:40.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f2ffd700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05f41989a0 0x7f05f419cce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:40.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.087+0000 7f05f2ffd700 1 -- 172.21.15.79:0/3501365071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e800f900 con 0x7f05dc0057d0 2024-05-08T22:30:40.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.088+0000 7f05f2ffd700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc0057d0 0x7f05f4198460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f05e8010010 tx=0x7f05e8004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:40.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.088+0000 7f05dbfff700 1 -- 172.21.15.79:0/3501365071 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f05e8013b00 con 0x7f05dc0057d0 2024-05-08T22:30:40.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.088+0000 7f05dbfff700 1 -- 172.21.15.79:0/3501365071 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05e80044f0 con 0x7f05dc0057d0 2024-05-08T22:30:40.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.088+0000 7f05f9272700 1 -- 172.21.15.79:0/3501365071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f419d280 con 0x7f05dc0057d0 2024-05-08T22:30:40.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.088+0000 7f05dbfff700 1 -- 172.21.15.79:0/3501365071 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f05e8003ca0 con 0x7f05dc0057d0 2024-05-08T22:30:40.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.088+0000 7f05f9272700 1 -- 172.21.15.79:0/3501365071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f419d7d0 con 0x7f05dc0057d0 2024-05-08T22:30:40.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.089+0000 7f05dbfff700 1 -- 172.21.15.79:0/3501365071 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f05e8011540 con 0x7f05dc0057d0 2024-05-08T22:30:40.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.089+0000 7f05dbfff700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f05e00385e0 0x7f05e003aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:40.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.089+0000 7f05dbfff700 1 -- 172.21.15.79:0/3501365071 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f05e804dcf0 con 0x7f05dc0057d0 2024-05-08T22:30:40.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.089+0000 7f05f9272700 1 -- 172.21.15.79:0/3501365071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f40fe030 con 0x7f05dc0057d0 2024-05-08T22:30:40.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.089+0000 7f05f27fc700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f05e00385e0 0x7f05e003aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:40.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.090+0000 7f05f27fc700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f05e00385e0 0x7f05e003aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009850 tx=0x7f05e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:40.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.095+0000 7f05dbfff700 1 -- 172.21.15.79:0/3501365071 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f05e8024400 con 0x7f05dc0057d0 2024-05-08T22:30:40.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.365+0000 7f05f9272700 1 -- 172.21.15.79:0/3501365071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f05f419daa0 con 0x7f05dc0057d0 2024-05-08T22:30:40.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.366+0000 7f05dbfff700 1 -- 172.21.15.79:0/3501365071 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1002 (secure 0 0 0) 0x7f05e8023360 con 0x7f05dc0057d0 2024-05-08T22:30:40.366 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:30:40.367 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":2,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","modified":"2024-05-08T22:30:34.919256Z","created":"2024-05-08T22:29:13.064164Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T22:30:40.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.368+0000 7f05f9272700 1 -- 172.21.15.79:0/3501365071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f05e00385e0 msgr2=0x7f05e003aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:40.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.368+0000 7f05f9272700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f05e00385e0 0x7f05e003aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009850 tx=0x7f05e4008040 comp rx=0 tx=0).stop 2024-05-08T22:30:40.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.368+0000 7f05f9272700 1 -- 172.21.15.79:0/3501365071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc0057d0 msgr2=0x7f05f4198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:40.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.368+0000 7f05f9272700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc0057d0 0x7f05f4198460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f05e8010010 tx=0x7f05e8004f80 comp rx=0 tx=0).stop 2024-05-08T22:30:40.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.368+0000 7f05f9272700 1 -- 172.21.15.79:0/3501365071 shutdown_connections 2024-05-08T22:30:40.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.368+0000 7f05f9272700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f05e00385e0 0x7f05e003aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:40.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.368+0000 7f05f9272700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05dc0057d0 0x7f05f4198460 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:40.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.368+0000 7f05f9272700 1 --2- 172.21.15.79:0/3501365071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05f41989a0 0x7f05f419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:40.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.368+0000 7f05f9272700 1 -- 172.21.15.79:0/3501365071 >> 172.21.15.79:0/3501365071 conn(0x7f05f4076570 msgr2=0x7f05f4102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:40.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.368+0000 7f05f9272700 1 -- 172.21.15.79:0/3501365071 shutdown_connections 2024-05-08T22:30:40.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:40.368+0000 7f05f9272700 1 -- 172.21.15.79:0/3501365071 wait complete. 2024-05-08T22:30:40.370 INFO:teuthology.orchestra.run.smithi079.stderr:dumped monmap epoch 2 2024-05-08T22:30:40.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:40.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:30:40.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: mon.a calling monitor election 2024-05-08T22:30:40.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:40.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: mon.c calling monitor election 2024-05-08T22:30:40.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: monmap e2: 2 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: fsmap 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: mgrmap e12: x(active, since 36s) 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: overall HEALTH_OK 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:40.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:40.391 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:40.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:40.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:30:40.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: mon.a calling monitor election 2024-05-08T22:30:40.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:40.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: mon.c calling monitor election 2024-05-08T22:30:40.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: monmap e2: 2 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: fsmap 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: mgrmap e12: x(active, since 36s) 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: overall HEALTH_OK 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:40.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:41.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:41 smithi079 ceph-mon[46738]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:41.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:41 smithi079 ceph-mon[46738]: Deploying daemon mon.b on smithi066 2024-05-08T22:30:41.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:41 smithi079 ceph-mon[46738]: from='client.? 172.21.15.79:0/3501365071' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:30:41.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:41 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:41.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:41 smithi062 ceph-mon[42571]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:41.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:41 smithi062 ceph-mon[42571]: Deploying daemon mon.b on smithi066 2024-05-08T22:30:41.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:41 smithi062 ceph-mon[42571]: from='client.? 172.21.15.79:0/3501365071' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:30:41.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:41 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:41.758 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T22:30:41.758 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T22:30:41.985 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:30:42.172 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:30:41 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:30:41.918+0000 7f67c5f5f700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T22:30:42.258 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 systemd[1]: Starting Ceph mon.b for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:30:42.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 bash[46793]: 76f1bfc1bd65f52790a5ea6f2eb99a463b9e3ab364a4190acc9f6873c8463dae 2024-05-08T22:30:42.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 systemd[1]: Started Ceph mon.b for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:30:42.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable), process ceph-mon, pid 7 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: pidfile_write: ignore empty --pid-file 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: load: jerasure load: lrc 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: RocksDB version: 7.9.2 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Git sha 0 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Compile date 2024-05-08 17:55:50 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: DB SUMMARY 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: DB Session ID: RHUOP7WKF0HSTVHFPQHG 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: CURRENT file: CURRENT 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-08T22:30:42.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.error_if_exists: 0 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.create_if_missing: 0 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.paranoid_checks: 1 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.env: 0x56276c70f820 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.info_log: 0x56276e01f520 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.statistics: (nil) 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.use_fsync: 0 2024-05-08T22:30:42.579 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_log_file_size: 0 2024-05-08T22:30:42.580 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T22:30:42.580 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T22:30:42.580 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T22:30:42.580 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T22:30:42.580 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.allow_fallocate: 1 2024-05-08T22:30:42.580 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T22:30:42.580 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T22:30:42.580 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.use_direct_reads: 0 2024-05-08T22:30:42.580 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T22:30:42.580 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T22:30:42.580 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.db_log_dir: 2024-05-08T22:30:42.582 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.wal_dir: 2024-05-08T22:30:42.582 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T22:30:42.582 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T22:30:42.582 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T22:30:42.582 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T22:30:42.582 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T22:30:42.582 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T22:30:42.582 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.write_buffer_manager: 0x56276d2b4320 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.unordered_write: 0 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T22:30:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.row_cache: None 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.wal_filter: None 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.two_write_queues: 0 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.wal_compression: 0 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.atomic_flush: 0 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T22:30:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.log_readahead_size: 0 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_background_jobs: 2 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_background_compactions: -1 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_subcompactions: 1 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T22:30:42.585 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_open_files: -1 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_background_flushes: -1 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Compression algorithms supported: 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T22:30:42.586 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: kZSTD supported: 0 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: kXpressCompression supported: 0 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: kZlibCompression supported: 1 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: kSnappyCompression supported: 1 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: kLZ4Compression supported: 1 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: kBZip2Compression supported: 0 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.merge_operator: 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_filter: None 2024-05-08T22:30:42.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56276e01f620) 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout: cache_index_and_filter_blocks: 1 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout: pin_top_level_index_and_filter: 1 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout: index_type: 0 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout: data_block_index_type: 0 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout: index_shortening: 1 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T22:30:42.588 INFO:journalctl@ceph.mon.b.smithi066.stdout: checksum: 4 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: no_block_cache: 0 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_cache: 0x56276d3371f0 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_cache_name: BinnedLRUCache 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_cache_options: 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: capacity : 536870912 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: num_shard_bits : 4 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: strict_capacity_limit : 0 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: high_pri_pool_ratio: 0.000 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_cache_compressed: (nil) 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: persistent_cache: (nil) 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_size: 4096 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_size_deviation: 10 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_restart_interval: 16 2024-05-08T22:30:42.589 INFO:journalctl@ceph.mon.b.smithi066.stdout: index_block_restart_interval: 1 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: metadata_block_size: 4096 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: partition_filters: 0 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: use_delta_encoding: 1 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: filter_policy: bloomfilter 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: whole_key_filtering: 1 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: verify_compression: 0 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: read_amp_bytes_per_bit: 0 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: format_version: 5 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: enable_index_compression: 1 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: block_align: 0 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: max_auto_readahead_size: 262144 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: prepopulate_block_cache: 0 2024-05-08T22:30:42.590 INFO:journalctl@ceph.mon.b.smithi066.stdout: initial_auto_readahead_size: 8192 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compression: NoCompression 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.num_levels: 7 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T22:30:42.591 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T22:30:42.592 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T22:30:42.593 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T22:30:42.594 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T22:30:42.594 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T22:30:42.594 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T22:30:42.594 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T22:30:42.594 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T22:30:42.594 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T22:30:42.594 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T22:30:42.594 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T22:30:42.594 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T22:30:42.594 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T22:30:42.594 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.table_properties_collectors: 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.inplace_update_support: 0 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T22:30:42.595 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T22:30:42.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T22:30:42.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.bloom_locality: 0 2024-05-08T22:30:42.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.max_successive_merges: 0 2024-05-08T22:30:42.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T22:30:42.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T22:30:42.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T22:30:42.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T22:30:42.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.ttl: 2592000 2024-05-08T22:30:42.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T22:30:42.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.enable_blob_files: false 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.min_blob_size: 0 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T22:30:42.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5a0f9c02-34f7-4285-a45b-b6aa7ef25482 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715207442462682, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715207442463150, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715207442, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5a0f9c02-34f7-4285-a45b-b6aa7ef25482", "db_session_id": "RHUOP7WKF0HSTVHFPQHG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715207442463219, "job": 1, "event": "recovery_finished"} 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56276d3d4000 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: DB pointer 0x56276d3c2000 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout: ** DB Stats ** 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:30:42.599 INFO:journalctl@ceph.mon.b.smithi066.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: ** Compaction Stats [default] ** 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-05-08T22:30:42.600 INFO:journalctl@ceph.mon.b.smithi066.stdout: ** Compaction Stats [default] ** 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:30:42.601 INFO:journalctl@ceph.mon.b.smithi066.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout: Block cache BinnedLRUCache@0x56276d3371f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: using public_addr v2:172.21.15.66:0/0 -> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: starting mon.b rank -1 at public addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] at bind addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(???) e0 preinit fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).mds e1 new map 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout: e1 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T22:30:42.602 INFO:journalctl@ceph.mon.b.smithi066.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout: legacy client fscid: -1 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout: 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout: No filesystems configured 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T22:30:42.603 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:42.604 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: Deploying daemon mon.b on smithi066 2024-05-08T22:30:42.604 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: from='client.? 172.21.15.79:0/3501365071' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:30:42.604 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:42.604 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:42 smithi066 ceph-mon[46830]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T22:30:47.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:47.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:30:47.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: mon.a calling monitor election 2024-05-08T22:30:47.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: mon.c calling monitor election 2024-05-08T22:30:47.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:47.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:47.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: mon.b calling monitor election 2024-05-08T22:30:47.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:47.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T22:30:47.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:30:47.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: fsmap 2024-05-08T22:30:47.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:30:47.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: mgrmap e12: x(active, since 43s) 2024-05-08T22:30:47.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: overall HEALTH_OK 2024-05-08T22:30:47.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:47.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:30:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: mon.a calling monitor election 2024-05-08T22:30:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: mon.c calling monitor election 2024-05-08T22:30:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: mon.b calling monitor election 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: fsmap 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: mgrmap e12: x(active, since 43s) 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: overall HEALTH_OK 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: mon.a calling monitor election 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: mon.c calling monitor election 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: mon.b calling monitor election 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:48.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T22:30:48.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T22:30:48.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: fsmap 2024-05-08T22:30:48.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T22:30:48.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: mgrmap e12: x(active, since 43s) 2024-05-08T22:30:48.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: overall HEALTH_OK 2024-05-08T22:30:48.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:48 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:48 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:48 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:48.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:48 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:48 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:48 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:48 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:48 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:48.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:48 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:48.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:48 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:48.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.916+0000 7f347577a700 1 -- 172.21.15.79:0/1734824880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f345c0062a0 msgr2=0x7f345c006680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:48.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.916+0000 7f347577a700 1 --2- 172.21.15.79:0/1734824880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f345c0062a0 0x7f345c006680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f34701009a0 tx=0x7f345800fe90 comp rx=0 tx=0).stop 2024-05-08T22:30:48.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.917+0000 7f347577a700 1 -- 172.21.15.79:0/1734824880 shutdown_connections 2024-05-08T22:30:48.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.917+0000 7f347577a700 1 --2- 172.21.15.79:0/1734824880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f345c0062a0 0x7f345c006680 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:48.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.917+0000 7f347577a700 1 -- 172.21.15.79:0/1734824880 >> 172.21.15.79:0/1734824880 conn(0x7f34700fb450 msgr2=0x7f34700fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:48.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.917+0000 7f347577a700 1 -- 172.21.15.79:0/1734824880 shutdown_connections 2024-05-08T22:30:48.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.917+0000 7f347577a700 1 -- 172.21.15.79:0/1734824880 wait complete. 2024-05-08T22:30:48.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.917+0000 7f347577a700 1 Processor -- start 2024-05-08T22:30:48.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f347577a700 1 -- start start 2024-05-08T22:30:48.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f347577a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f345c0062a0 0x7f3470196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:48.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f347577a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34701969b0 0x7f347019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:48.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f347577a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f347019b2a0 0x7f347019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:48.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f347577a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34701970a0 con 0x7f347019b2a0 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f347577a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34701a3010 con 0x7f34701969b0 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f347577a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34701a3150 con 0x7f345c0062a0 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f345c0062a0 0x7f3470196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f345c0062a0 0x7f3470196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:55122/0 (socket says 172.21.15.79:55122) 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346effd700 1 -- 172.21.15.79:0/3969208277 learned_addr learned my addr 172.21.15.79:0/3969208277 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346e7fc700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34701969b0 0x7f347019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346f7fe700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f347019b2a0 0x7f347019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346effd700 1 -- 172.21.15.79:0/3969208277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f345c0062a0 msgr2=0x7f3470196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346effd700 1 -- 172.21.15.79:0/3969208277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f345c0062a0 msgr2=0x7f3470196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346effd700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f345c0062a0 0x7f3470196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346effd700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f345c0062a0 0x7f3470196470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346e7fc700 1 -- 172.21.15.79:0/3969208277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34701969b0 msgr2=0x7f347019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T22:30:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346e7fc700 1 -- 172.21.15.79:0/3969208277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34701969b0 msgr2=0x7f347019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346e7fc700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34701969b0 0x7f347019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346e7fc700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34701969b0 0x7f347019ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346f7fe700 1 -- 172.21.15.79:0/3969208277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f345c0062a0 msgr2=0x7f3470196470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346f7fe700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f345c0062a0 0x7f3470196470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346f7fe700 1 -- 172.21.15.79:0/3969208277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34701969b0 msgr2=0x7f347019ad60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346f7fe700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34701969b0 0x7f347019ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.918+0000 7f346f7fe700 1 -- 172.21.15.79:0/3969208277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345800f8e0 con 0x7f347019b2a0 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.919+0000 7f346f7fe700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f347019b2a0 0x7f347019b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f346800bd50 tx=0x7f346800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.919+0000 7f3467fff700 1 -- 172.21.15.79:0/3969208277 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346800cd10 con 0x7f347019b2a0 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.919+0000 7f3467fff700 1 -- 172.21.15.79:0/3969208277 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f346800ce70 con 0x7f347019b2a0 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.919+0000 7f347577a700 1 -- 172.21.15.79:0/3969208277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34701a3430 con 0x7f347019b2a0 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.919+0000 7f3467fff700 1 -- 172.21.15.79:0/3969208277 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468012840 con 0x7f347019b2a0 2024-05-08T22:30:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.919+0000 7f347577a700 1 -- 172.21.15.79:0/3969208277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34701a38e0 con 0x7f347019b2a0 2024-05-08T22:30:48.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.920+0000 7f3467fff700 1 -- 172.21.15.79:0/3969208277 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f34680129c0 con 0x7f347019b2a0 2024-05-08T22:30:48.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.920+0000 7f347577a700 1 -- 172.21.15.79:0/3969208277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3470101020 con 0x7f347019b2a0 2024-05-08T22:30:48.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.920+0000 7f3467fff700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3450038c20 0x7f345003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:48.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.920+0000 7f3467fff700 1 -- 172.21.15.79:0/3969208277 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f346804ad90 con 0x7f347019b2a0 2024-05-08T22:30:48.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.920+0000 7f346effd700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3450038c20 0x7f345003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:48.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.921+0000 7f346effd700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3450038c20 0x7f345003b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3458005b40 tx=0x7f3458005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:48.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:48 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:48 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:48 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:48.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:48 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:48 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:48 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:48 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:48.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:48 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:48.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:48 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:48.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:48 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:48.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:48.925+0000 7f3467fff700 1 -- 172.21.15.79:0/3969208277 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f346804e030 con 0x7f347019b2a0 2024-05-08T22:30:49.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:48 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:48 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:48 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:49.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:48 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:48 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:48 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:48 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:48 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:49.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:48 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:49.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:48 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:30:49.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.205+0000 7f347577a700 1 -- 172.21.15.79:0/3969208277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f34700625d0 con 0x7f347019b2a0 2024-05-08T22:30:49.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.206+0000 7f3467fff700 1 -- 172.21.15.79:0/3969208277 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1268 (secure 0 0 0) 0x7f346801d3b0 con 0x7f347019b2a0 2024-05-08T22:30:49.206 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-05-08T22:30:49.207 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":3,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","modified":"2024-05-08T22:30:42.495922Z","created":"2024-05-08T22:29:13.064164Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-08T22:30:49.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 -- 172.21.15.79:0/3969208277 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3450038c20 msgr2=0x7f345003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:49.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3450038c20 0x7f345003b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3458005b40 tx=0x7f3458005a90 comp rx=0 tx=0).stop 2024-05-08T22:30:49.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 -- 172.21.15.79:0/3969208277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f347019b2a0 msgr2=0x7f347019b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:49.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f347019b2a0 0x7f347019b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f346800bd50 tx=0x7f346800be30 comp rx=0 tx=0).stop 2024-05-08T22:30:49.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 -- 172.21.15.79:0/3969208277 shutdown_connections 2024-05-08T22:30:49.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f347019b2a0 0x7f347019b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:49.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f345c0062a0 0x7f3470196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:49.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3450038c20 0x7f345003b0e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:49.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 --2- 172.21.15.79:0/3969208277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34701969b0 0x7f347019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:49.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 -- 172.21.15.79:0/3969208277 >> 172.21.15.79:0/3969208277 conn(0x7f34700fb450 msgr2=0x7f3470068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:49.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 -- 172.21.15.79:0/3969208277 shutdown_connections 2024-05-08T22:30:49.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:49.208+0000 7f347577a700 1 -- 172.21.15.79:0/3969208277 wait complete. 2024-05-08T22:30:49.211 INFO:teuthology.orchestra.run.smithi079.stderr:dumped monmap epoch 3 2024-05-08T22:30:49.593 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-08T22:30:49.594 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-08T22:30:49.774 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:49.774 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: Updating smithi062:/etc/ceph/ceph.conf 2024-05-08T22:30:49.774 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: Updating smithi066:/etc/ceph/ceph.conf 2024-05-08T22:30:49.774 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: Updating smithi079:/etc/ceph/ceph.conf 2024-05-08T22:30:49.775 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: Updating smithi062:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:49.775 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: Updating smithi066:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:49.775 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: Updating smithi079:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:49.775 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.775 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.775 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.775 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.775 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.775 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.776 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.776 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.776 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.776 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.776 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.776 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.776 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:49.776 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:30:49.776 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:49.776 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:49 smithi062 ceph-mon[42571]: from='client.? 172.21.15.79:0/3969208277' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:30:49.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:49.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: Updating smithi062:/etc/ceph/ceph.conf 2024-05-08T22:30:49.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: Updating smithi066:/etc/ceph/ceph.conf 2024-05-08T22:30:49.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: Updating smithi079:/etc/ceph/ceph.conf 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: Updating smithi062:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: Updating smithi066:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: Updating smithi079:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:49.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:49.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:30:49.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:49.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:49 smithi079 ceph-mon[46738]: from='client.? 172.21.15.79:0/3969208277' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:30:49.976 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:30:50.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: Updating smithi062:/etc/ceph/ceph.conf 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: Updating smithi066:/etc/ceph/ceph.conf 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: Updating smithi079:/etc/ceph/ceph.conf 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: Updating smithi062:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: Updating smithi066:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: Updating smithi079:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.076 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:50.076 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:50.076 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:30:50.076 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:50.076 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:49 smithi066 ceph-mon[46830]: from='client.? 172.21.15.79:0/3969208277' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T22:30:50.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.961+0000 7f42753cb700 1 -- 172.21.15.62:0/3477595691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4270068950 msgr2=0x7f4270068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.961+0000 7f42753cb700 1 --2- 172.21.15.62:0/3477595691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4270068950 0x7f4270068db0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4258009990 tx=0x7f4258009ca0 comp rx=0 tx=0).stop 2024-05-08T22:30:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.961+0000 7f42753cb700 1 -- 172.21.15.62:0/3477595691 shutdown_connections 2024-05-08T22:30:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.961+0000 7f42753cb700 1 --2- 172.21.15.62:0/3477595691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42700692f0 0x7f427010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.961+0000 7f42753cb700 1 --2- 172.21.15.62:0/3477595691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4270107020 0x7f4270107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.961+0000 7f42753cb700 1 --2- 172.21.15.62:0/3477595691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4270068950 0x7f4270068db0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.961+0000 7f42753cb700 1 -- 172.21.15.62:0/3477595691 >> 172.21.15.62:0/3477595691 conn(0x7f4270075f80 msgr2=0x7f4270076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.961+0000 7f42753cb700 1 -- 172.21.15.62:0/3477595691 shutdown_connections 2024-05-08T22:30:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.961+0000 7f42753cb700 1 -- 172.21.15.62:0/3477595691 wait complete. 2024-05-08T22:30:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.962+0000 7f42753cb700 1 Processor -- start 2024-05-08T22:30:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.962+0000 7f42753cb700 1 -- start start 2024-05-08T22:30:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.962+0000 7f42753cb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4270068950 0x7f4270197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:50.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.962+0000 7f42753cb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42700692f0 0x7f4270197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:50.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.962+0000 7f42753cb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4270107020 0x7f427019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:50.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f42753cb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427019c330 con 0x7f42700692f0 2024-05-08T22:30:50.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f42753cb700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427019c4a0 con 0x7f4270068950 2024-05-08T22:30:50.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f42753cb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427019c660 con 0x7f4270107020 2024-05-08T22:30:50.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42700692f0 0x7f4270197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:50.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42700692f0 0x7f4270197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50134/0 (socket says 172.21.15.62:50134) 2024-05-08T22:30:50.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426e7fc700 1 -- 172.21.15.62:0/2781631815 learned_addr learned my addr 172.21.15.62:0/2781631815 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:50.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426effd700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4270068950 0x7f4270197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:50.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426f7fe700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4270107020 0x7f427019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:50.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426e7fc700 1 -- 172.21.15.62:0/2781631815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4270107020 msgr2=0x7f427019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426e7fc700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4270107020 0x7f427019bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426e7fc700 1 -- 172.21.15.62:0/2781631815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4270068950 msgr2=0x7f4270197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426e7fc700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4270068950 0x7f4270197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426e7fc700 1 -- 172.21.15.62:0/2781631815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42580095f0 con 0x7f42700692f0 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426effd700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4270068950 0x7f4270197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f426e7fc700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42700692f0 0x7f4270197970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4258000c00 tx=0x7f4258004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f4267fff700 1 -- 172.21.15.62:0/2781631815 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425801d070 con 0x7f42700692f0 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f42753cb700 1 -- 172.21.15.62:0/2781631815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427019c8e0 con 0x7f42700692f0 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f4267fff700 1 -- 172.21.15.62:0/2781631815 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4258022470 con 0x7f42700692f0 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f4267fff700 1 -- 172.21.15.62:0/2781631815 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258017650 con 0x7f42700692f0 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.963+0000 7f42753cb700 1 -- 172.21.15.62:0/2781631815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42701a41a0 con 0x7f42700692f0 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.964+0000 7f4267fff700 1 -- 172.21.15.62:0/2781631815 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f4258005030 con 0x7f42700692f0 2024-05-08T22:30:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.965+0000 7f42753cb700 1 -- 172.21.15.62:0/2781631815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4270109540 con 0x7f42700692f0 2024-05-08T22:30:50.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.965+0000 7f4267fff700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f425c0389d0 0x7f425c03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:50.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.965+0000 7f4267fff700 1 -- 172.21.15.62:0/2781631815 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f425804c7d0 con 0x7f42700692f0 2024-05-08T22:30:50.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.965+0000 7f426effd700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f425c0389d0 0x7f425c03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:50.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.965+0000 7f426effd700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f425c0389d0 0x7f425c03ae90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4260011490 tx=0x7f4260009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:50.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:50.970+0000 7f4267fff700 1 -- 172.21.15.62:0/2781631815 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4258016bd0 con 0x7f42700692f0 2024-05-08T22:30:51.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:50 smithi066 ceph-mon[46830]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T22:30:51.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:50 smithi066 ceph-mon[46830]: Reconfiguring daemon mon.a on smithi062 2024-05-08T22:30:51.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:50 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:51.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:50 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:51.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:50 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:51.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:50 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:30:51.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:50 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:51.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:50 smithi062 ceph-mon[42571]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T22:30:51.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:50 smithi062 ceph-mon[42571]: Reconfiguring daemon mon.a on smithi062 2024-05-08T22:30:51.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:50 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:51.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:50 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:51.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:50 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:51.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:50 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:30:51.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:50 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.172+0000 7f42753cb700 1 -- 172.21.15.62:0/2781631815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f42701026d0 con 0x7f42700692f0 2024-05-08T22:30:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.173+0000 7f4267fff700 1 -- 172.21.15.62:0/2781631815 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7f4258027070 con 0x7f42700692f0 2024-05-08T22:30:51.173 INFO:teuthology.orchestra.run.smithi062.stdout:# minimal ceph.conf for 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:30:51.174 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-05-08T22:30:51.174 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:30:51.174 INFO:teuthology.orchestra.run.smithi062.stdout: mon_host = [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] 2024-05-08T22:30:51.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 -- 172.21.15.62:0/2781631815 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f425c0389d0 msgr2=0x7f425c03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:51.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f425c0389d0 0x7f425c03ae90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4260011490 tx=0x7f4260009260 comp rx=0 tx=0).stop 2024-05-08T22:30:51.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 -- 172.21.15.62:0/2781631815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42700692f0 msgr2=0x7f4270197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:51.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42700692f0 0x7f4270197970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4258000c00 tx=0x7f4258004730 comp rx=0 tx=0).stop 2024-05-08T22:30:51.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 -- 172.21.15.62:0/2781631815 shutdown_connections 2024-05-08T22:30:51.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4270107020 0x7f427019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:51.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f425c0389d0 0x7f425c03ae90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:51.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4270068950 0x7f4270197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:51.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 --2- 172.21.15.62:0/2781631815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42700692f0 0x7f4270197970 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:51.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 -- 172.21.15.62:0/2781631815 >> 172.21.15.62:0/2781631815 conn(0x7f4270075f80 msgr2=0x7f42700fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:51.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 -- 172.21.15.62:0/2781631815 shutdown_connections 2024-05-08T22:30:51.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:51.175+0000 7f42753cb700 1 -- 172.21.15.62:0/2781631815 wait complete. 2024-05-08T22:30:51.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:50 smithi079 ceph-mon[46738]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T22:30:51.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:50 smithi079 ceph-mon[46738]: Reconfiguring daemon mon.a on smithi062 2024-05-08T22:30:51.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:50 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:51.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:50 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:51.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:50 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:51.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:50 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:30:51.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:50 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:51.570 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-08T22:30:51.571 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:30:51.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T22:30:51.602 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:30:51.602 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:51.672 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-08T22:30:51.672 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T22:30:51.707 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-08T22:30:51.707 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:51.739 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-08T22:30:51.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T22:30:51.770 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-08T22:30:51.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T22:30:51.840 INFO:tasks.cephadm:Adding mgr.x on smithi062 2024-05-08T22:30:51.841 INFO:tasks.cephadm:Adding mgr.y on smithi066 2024-05-08T22:30:51.841 INFO:tasks.cephadm:Adding mgr.z on smithi079 2024-05-08T22:30:51.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi062=x;smithi066=y;smithi079=z' 2024-05-08T22:30:52.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:51 smithi066 ceph-mon[46830]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:52.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:51 smithi066 ceph-mon[46830]: Reconfiguring mon.b (monmap changed)... 2024-05-08T22:30:52.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:51 smithi066 ceph-mon[46830]: Reconfiguring daemon mon.b on smithi066 2024-05-08T22:30:52.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:52.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:52.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:52.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:30:52.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:52.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:51 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2781631815' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:52.099 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:30:52.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:51 smithi079 ceph-mon[46738]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:52.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:51 smithi079 ceph-mon[46738]: Reconfiguring mon.b (monmap changed)... 2024-05-08T22:30:52.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:51 smithi079 ceph-mon[46738]: Reconfiguring daemon mon.b on smithi066 2024-05-08T22:30:52.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:52.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:52.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:52.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:30:52.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:52.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:51 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2781631815' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:52.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:51 smithi062 ceph-mon[42571]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:52.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:51 smithi062 ceph-mon[42571]: Reconfiguring mon.b (monmap changed)... 2024-05-08T22:30:52.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:51 smithi062 ceph-mon[42571]: Reconfiguring daemon mon.b on smithi066 2024-05-08T22:30:52.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:52.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:52.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T22:30:52.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T22:30:52.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:52.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:51 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2781631815' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:53.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.107+0000 7f8fa99ea700 1 -- 172.21.15.79:0/2470589604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fa4101aa0 msgr2=0x7f8fa4103e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.107+0000 7f8fa99ea700 1 --2- 172.21.15.79:0/2470589604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fa4101aa0 0x7f8fa4103e90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009990 tx=0x7f8f98009ca0 comp rx=0 tx=0).stop 2024-05-08T22:30:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.107+0000 7f8fa99ea700 1 -- 172.21.15.79:0/2470589604 shutdown_connections 2024-05-08T22:30:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.107+0000 7f8fa99ea700 1 --2- 172.21.15.79:0/2470589604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa4106d00 0x7f8fa4109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.107+0000 7f8fa99ea700 1 --2- 172.21.15.79:0/2470589604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fa4101aa0 0x7f8fa4103e90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.107+0000 7f8fa99ea700 1 --2- 172.21.15.79:0/2470589604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa41043d0 0x7f8fa41067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.107+0000 7f8fa99ea700 1 -- 172.21.15.79:0/2470589604 >> 172.21.15.79:0/2470589604 conn(0x7f8fa40fb490 msgr2=0x7f8fa40fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.107+0000 7f8fa99ea700 1 -- 172.21.15.79:0/2470589604 shutdown_connections 2024-05-08T22:30:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.107+0000 7f8fa99ea700 1 -- 172.21.15.79:0/2470589604 wait complete. 2024-05-08T22:30:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa99ea700 1 Processor -- start 2024-05-08T22:30:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa99ea700 1 -- start start 2024-05-08T22:30:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa99ea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa4101aa0 0x7f8fa4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa99ea700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa41043d0 0x7f8fa41974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa99ea700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fa4106d00 0x7f8fa419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa99ea700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa419ead0 con 0x7f8fa4101aa0 2024-05-08T22:30:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa99ea700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa419ec10 con 0x7f8fa41043d0 2024-05-08T22:30:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa99ea700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa419eda0 con 0x7f8fa4106d00 2024-05-08T22:30:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fa4106d00 0x7f8fa419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fa4106d00 0x7f8fa419e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52230/0 (socket says 172.21.15.79:52230) 2024-05-08T22:30:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa37fe700 1 -- 172.21.15.79:0/3120040974 learned_addr learned my addr 172.21.15.79:0/3120040974 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-08T22:30:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa2ffd700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa4101aa0 0x7f8fa4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa27fc700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa41043d0 0x7f8fa41974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa37fe700 1 -- 172.21.15.79:0/3120040974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa41043d0 msgr2=0x7f8fa41974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa37fe700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa41043d0 0x7f8fa41974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa37fe700 1 -- 172.21.15.79:0/3120040974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa4101aa0 msgr2=0x7f8fa4196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa37fe700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa4101aa0 0x7f8fa4196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa37fe700 1 -- 172.21.15.79:0/3120040974 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980095f0 con 0x7f8fa4106d00 2024-05-08T22:30:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.108+0000 7f8fa27fc700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa41043d0 0x7f8fa41974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:30:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.109+0000 7f8fa37fe700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fa4106d00 0x7f8fa419e500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00ba10 tx=0x7f8f9c00bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.109+0000 7f8fa89e8700 1 -- 172.21.15.79:0/3120040974 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00c930 con 0x7f8fa4106d00 2024-05-08T22:30:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.109+0000 7f8fa89e8700 1 -- 172.21.15.79:0/3120040974 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f9c014440 con 0x7f8fa4106d00 2024-05-08T22:30:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.109+0000 7f8fa99ea700 1 -- 172.21.15.79:0/3120040974 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa419f080 con 0x7f8fa4106d00 2024-05-08T22:30:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.109+0000 7f8fa89e8700 1 -- 172.21.15.79:0/3120040974 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0125a0 con 0x7f8fa4106d00 2024-05-08T22:30:53.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.109+0000 7f8fa99ea700 1 -- 172.21.15.79:0/3120040974 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa419f580 con 0x7f8fa4106d00 2024-05-08T22:30:53.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.110+0000 7f8fa89e8700 1 -- 172.21.15.79:0/3120040974 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f8f9c012700 con 0x7f8fa4106d00 2024-05-08T22:30:53.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.110+0000 7f8fa89e8700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f8c0389d0 0x7f8f8c03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:53.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.110+0000 7f8fa89e8700 1 -- 172.21.15.79:0/3120040974 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8f9c006e40 con 0x7f8fa4106d00 2024-05-08T22:30:53.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.110+0000 7f8fa99ea700 1 -- 172.21.15.79:0/3120040974 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa40fd010 con 0x7f8fa4106d00 2024-05-08T22:30:53.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.110+0000 7f8fa2ffd700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f8c0389d0 0x7f8f8c03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:53.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.110+0000 7f8fa2ffd700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f8c0389d0 0x7f8f8c03ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800b5c0 tx=0x7f8f98003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:53.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.115+0000 7f8fa89e8700 1 -- 172.21.15.79:0/3120040974 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8f9c01b8b0 con 0x7f8fa4106d00 2024-05-08T22:30:53.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:52 smithi062 ceph-mon[42571]: Reconfiguring mon.c (monmap changed)... 2024-05-08T22:30:53.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:52 smithi062 ceph-mon[42571]: Reconfiguring daemon mon.c on smithi079 2024-05-08T22:30:53.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:53.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:53.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:53.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:53.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:53.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:53.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.318+0000 7f8fa99ea700 1 -- 172.21.15.79:0/3120040974 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi066=y;smithi079=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fa40613e0 con 0x7f8f8c0389d0 2024-05-08T22:30:53.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:52 smithi066 ceph-mon[46830]: Reconfiguring mon.c (monmap changed)... 2024-05-08T22:30:53.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:52 smithi066 ceph-mon[46830]: Reconfiguring daemon mon.c on smithi079 2024-05-08T22:30:53.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:53.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:53.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:53.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:53.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:53.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:53.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.325+0000 7f8fa89e8700 1 -- 172.21.15.79:0/3120040974 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8fa40613e0 con 0x7f8f8c0389d0 2024-05-08T22:30:53.325 INFO:teuthology.orchestra.run.smithi079.stdout:Scheduled mgr update... 2024-05-08T22:30:53.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 -- 172.21.15.79:0/3120040974 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f8c0389d0 msgr2=0x7f8f8c03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:53.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f8c0389d0 0x7f8f8c03ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800b5c0 tx=0x7f8f98003790 comp rx=0 tx=0).stop 2024-05-08T22:30:53.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 -- 172.21.15.79:0/3120040974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fa4106d00 msgr2=0x7f8fa419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:53.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fa4106d00 0x7f8fa419e500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00ba10 tx=0x7f8f9c00bd20 comp rx=0 tx=0).stop 2024-05-08T22:30:53.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 -- 172.21.15.79:0/3120040974 shutdown_connections 2024-05-08T22:30:53.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fa4106d00 0x7f8fa419e500 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:53.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa4101aa0 0x7f8fa4196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:53.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f8c0389d0 0x7f8f8c03ae90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:53.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 --2- 172.21.15.79:0/3120040974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fa41043d0 0x7f8fa41974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:53.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 -- 172.21.15.79:0/3120040974 >> 172.21.15.79:0/3120040974 conn(0x7f8fa40fb490 msgr2=0x7f8fa41023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:53.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 -- 172.21.15.79:0/3120040974 shutdown_connections 2024-05-08T22:30:53.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:30:53.328+0000 7f8fa99ea700 1 -- 172.21.15.79:0/3120040974 wait complete. 2024-05-08T22:30:53.347 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:52 smithi079 ceph-mon[46738]: Reconfiguring mon.c (monmap changed)... 2024-05-08T22:30:53.348 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:52 smithi079 ceph-mon[46738]: Reconfiguring daemon mon.c on smithi079 2024-05-08T22:30:53.348 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:53.348 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:53.348 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:53.348 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:53.348 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:53.348 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:53.730 DEBUG:teuthology.orchestra.run.smithi066:mgr.y> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.y.service 2024-05-08T22:30:53.733 DEBUG:teuthology.orchestra.run.smithi079:mgr.z> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.z.service 2024-05-08T22:30:53.735 INFO:tasks.cephadm:Deploying OSDs... 2024-05-08T22:30:53.735 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:30:53.735 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T22:30:53.757 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T22:30:53.758 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-05-08T22:30:53.767 INFO:journalctl@ceph.mgr.z.smithi079.stdout:-- Logs begin at Wed 2024-05-08 22:12:11 UTC. -- 2024-05-08T22:30:53.767 INFO:journalctl@ceph.mgr.y.smithi066.stdout:-- Logs begin at Wed 2024-05-08 22:13:17 UTC. -- 2024-05-08T22:30:53.819 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T22:30:53.819 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:53.819 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 183499 Links: 1 2024-05-08T22:30:53.820 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:53.820 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:53.820 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-05-08 22:30:14.151947451 +0000 2024-05-08T22:30:53.820 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-05-08 22:23:46.417535435 +0000 2024-05-08T22:30:53.820 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-05-08 22:23:46.417535435 +0000 2024-05-08T22:30:53.820 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-05-08T22:30:53.820 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T22:30:53.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-05-08T22:30:53.891 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-05-08T22:30:53.891 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000202077 s, 2.5 MB/s 2024-05-08T22:30:53.892 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T22:30:53.954 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-05-08T22:30:54.013 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T22:30:54.014 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:54.014 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 181725 Links: 1 2024-05-08T22:30:54.014 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:54.014 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:54.014 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-05-08 22:30:14.211946530 +0000 2024-05-08T22:30:54.014 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-05-08 22:23:46.773530062 +0000 2024-05-08T22:30:54.015 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-05-08 22:23:46.773530062 +0000 2024-05-08T22:30:54.015 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-05-08T22:30:54.015 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T22:30:54.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:53 smithi079 ceph-mon[46738]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:54.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:54.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:54.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:54.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:54.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:54.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:30:54.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:30:54.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:30:54.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:54.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:53 smithi066 ceph-mon[46830]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:54.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:54.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:54.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:54.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:54.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:54.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:30:54.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:30:54.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:30:54.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:54.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-05-08T22:30:54.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-05-08T22:30:54.085 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000236898 s, 2.2 MB/s 2024-05-08T22:30:54.086 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T22:30:54.146 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-05-08T22:30:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:53 smithi062 ceph-mon[42571]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:30:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:30:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:30:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:54.191 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T22:30:54.191 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:54.191 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 181836 Links: 1 2024-05-08T22:30:54.191 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:54.191 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:54.191 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-05-08 22:30:14.273945579 +0000 2024-05-08T22:30:54.191 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-05-08 22:23:47.125524750 +0000 2024-05-08T22:30:54.192 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-05-08 22:23:47.125524750 +0000 2024-05-08T22:30:54.192 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-05-08T22:30:54.192 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T22:30:54.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-05-08T22:30:54.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-05-08T22:30:54.258 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000214536 s, 2.4 MB/s 2024-05-08T22:30:54.259 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T22:30:54.319 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-05-08T22:30:54.377 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T22:30:54.377 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:54.377 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 183680 Links: 1 2024-05-08T22:30:54.377 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:54.377 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:54.378 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-05-08 22:30:14.341944536 +0000 2024-05-08T22:30:54.378 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-05-08 22:23:47.481519377 +0000 2024-05-08T22:30:54.378 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-05-08 22:23:47.481519377 +0000 2024-05-08T22:30:54.378 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-05-08T22:30:54.379 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T22:30:54.444 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-05-08T22:30:54.444 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-05-08T22:30:54.444 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000234512 s, 2.2 MB/s 2024-05-08T22:30:54.445 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T22:30:54.503 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-08T22:30:54.503 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T22:30:54.526 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T22:30:54.527 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-05-08T22:30:54.588 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T22:30:54.588 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:54.589 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 181080 Links: 1 2024-05-08T22:30:54.589 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:54.589 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:54.589 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-05-08 22:30:42.066111150 +0000 2024-05-08T22:30:54.589 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-05-08 22:23:53.515423690 +0000 2024-05-08T22:30:54.589 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-05-08 22:23:53.515423690 +0000 2024-05-08T22:30:54.589 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: - 2024-05-08T22:30:54.589 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T22:30:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-05-08T22:30:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-05-08T22:30:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000243518 s, 2.1 MB/s 2024-05-08T22:30:54.662 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T22:30:54.725 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-05-08T22:30:54.785 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T22:30:54.785 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:54.786 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 183778 Links: 1 2024-05-08T22:30:54.786 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:54.786 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:54.786 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-05-08 22:30:42.066111150 +0000 2024-05-08T22:30:54.786 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-05-08 22:23:53.861421331 +0000 2024-05-08T22:30:54.786 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-05-08 22:23:53.861421331 +0000 2024-05-08T22:30:54.786 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: - 2024-05-08T22:30:54.786 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T22:30:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-05-08T22:30:54.855 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-05-08T22:30:54.856 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000222004 s, 2.3 MB/s 2024-05-08T22:30:54.857 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T22:30:54.888 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:54 smithi079 systemd[1]: Starting Ceph mgr.z for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:30:54.917 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-05-08T22:30:54.977 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T22:30:54.977 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:54.977 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 178643 Links: 1 2024-05-08T22:30:54.977 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:54.977 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:54.977 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-05-08 22:30:42.066111150 +0000 2024-05-08T22:30:54.977 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-05-08 22:23:54.208418964 +0000 2024-05-08T22:30:54.978 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-05-08 22:23:54.208418964 +0000 2024-05-08T22:30:54.978 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: - 2024-05-08T22:30:54.978 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T22:30:55.046 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-05-08T22:30:55.046 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-05-08T22:30:55.046 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.00019797 s, 2.6 MB/s 2024-05-08T22:30:55.047 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T22:30:55.107 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-05-08T22:30:55.166 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T22:30:55.166 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:55.166 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 179575 Links: 1 2024-05-08T22:30:55.167 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:55.167 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:55.167 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-05-08 22:30:42.067111142 +0000 2024-05-08T22:30:55.167 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-05-08 22:23:54.548416645 +0000 2024-05-08T22:30:55.167 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-05-08 22:23:54.548416645 +0000 2024-05-08T22:30:55.167 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: - 2024-05-08T22:30:55.167 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T22:30:55.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:54 smithi062 ceph-mon[42571]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi066=y;smithi079=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:55.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:54 smithi062 ceph-mon[42571]: Saving service mgr spec with placement smithi062=x;smithi066=y;smithi079=z;count:3 2024-05-08T22:30:55.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:54 smithi062 ceph-mon[42571]: Deploying daemon mgr.z on smithi079 2024-05-08T22:30:55.232 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:54 smithi066 ceph-mon[46830]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi066=y;smithi079=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:55.232 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:54 smithi066 ceph-mon[46830]: Saving service mgr spec with placement smithi062=x;smithi066=y;smithi079=z;count:3 2024-05-08T22:30:55.232 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:54 smithi066 ceph-mon[46830]: Deploying daemon mgr.z on smithi079 2024-05-08T22:30:55.234 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-05-08T22:30:55.234 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-05-08T22:30:55.234 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000243006 s, 2.1 MB/s 2024-05-08T22:30:55.236 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T22:30:55.293 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-05-08T22:30:55.293 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T22:30:55.312 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T22:30:55.312 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2024-05-08T22:30:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T22:30:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:55.369 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 182472 Links: 1 2024-05-08T22:30:55.370 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:55.370 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:55.370 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-05-08 22:30:54.646613665 +0000 2024-05-08T22:30:55.370 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-05-08 22:23:36.371418570 +0000 2024-05-08T22:30:55.370 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-05-08 22:23:36.371418570 +0000 2024-05-08T22:30:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-05-08T22:30:55.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T22:30:55.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:54 smithi079 ceph-mon[46738]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi066=y;smithi079=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:30:55.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:54 smithi079 ceph-mon[46738]: Saving service mgr spec with placement smithi062=x;smithi066=y;smithi079=z;count:3 2024-05-08T22:30:55.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:54 smithi079 ceph-mon[46738]: Deploying daemon mgr.z on smithi079 2024-05-08T22:30:55.389 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:55 smithi079 ceph-mgr[48626]: -- 172.21.15.79:0/3889145304 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a21f3d61a0 con 0x55a21f3acc00 2024-05-08T22:30:55.389 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:55 smithi079 bash[48588]: 6e76e2e671e5a8cdff656aa04d2df56d12d03c50a17ad630538d73bdaa72c14c 2024-05-08T22:30:55.389 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:55 smithi079 systemd[1]: Started Ceph mgr.z for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:30:55.417 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-05-08T22:30:55.417 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-05-08T22:30:55.417 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000229178 s, 2.2 MB/s 2024-05-08T22:30:55.418 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T22:30:55.476 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2024-05-08T22:30:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T22:30:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 182544 Links: 1 2024-05-08T22:30:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-05-08 22:30:54.646613665 +0000 2024-05-08T22:30:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-05-08 22:23:36.718410871 +0000 2024-05-08T22:30:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-05-08 22:23:36.718410871 +0000 2024-05-08T22:30:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-05-08T22:30:55.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T22:30:55.597 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:55 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:55.501+0000 7f8a726a5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:30:55.600 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-05-08T22:30:55.600 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-05-08T22:30:55.600 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000200328 s, 2.6 MB/s 2024-05-08T22:30:55.602 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T22:30:55.660 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2024-05-08T22:30:55.688 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:55 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:55.597+0000 7f8a726a5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:30:55.688 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:55 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:55.688+0000 7f8a726a5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:30:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T22:30:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:55.705 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 179849 Links: 1 2024-05-08T22:30:55.705 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:55.705 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:55.705 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-05-08 22:30:54.646613665 +0000 2024-05-08T22:30:55.705 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-05-08 22:23:37.072403017 +0000 2024-05-08T22:30:55.705 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-05-08 22:23:37.072403017 +0000 2024-05-08T22:30:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-05-08T22:30:55.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T22:30:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-05-08T22:30:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-05-08T22:30:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000221385 s, 2.3 MB/s 2024-05-08T22:30:55.773 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T22:30:55.831 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2024-05-08T22:30:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T22:30:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T22:30:55.889 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 179134 Links: 1 2024-05-08T22:30:55.889 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:30:55.889 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T22:30:55.889 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-05-08 22:30:54.646613665 +0000 2024-05-08T22:30:55.889 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-05-08 22:23:37.422395251 +0000 2024-05-08T22:30:55.889 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-05-08 22:23:37.422395251 +0000 2024-05-08T22:30:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-05-08T22:30:55.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T22:30:55.890 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:55 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:55.781+0000 7f8a726a5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:30:55.956 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-05-08T22:30:55.957 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-05-08T22:30:55.957 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000216235 s, 2.4 MB/s 2024-05-08T22:30:55.958 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T22:30:56.018 INFO:tasks.cephadm:Deploying osd.0 on smithi062 with /dev/vg_nvme/lv_4... 2024-05-08T22:30:56.018 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T22:30:56.169 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:55 smithi066 ceph-mon[46830]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:56.169 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:56.169 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:56.169 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:56.169 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:30:56.169 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:30:56.169 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:30:56.169 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:56.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:55 smithi062 ceph-mon[42571]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:56.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:56.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:56.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:56.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:30:56.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:30:56.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:30:56.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:30:56.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:55 smithi079 ceph-mon[46738]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:56.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:56.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:56.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:56.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:30:56.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T22:30:56.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:30:56.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:56.702 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:56 smithi066 systemd[1]: Starting Ceph mgr.y for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:30:56.767 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:56 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:56.434+0000 7f8a726a5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:30:56.767 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:56 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:56.683+0000 7f8a726a5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:30:57.034 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:56 smithi079 ceph-mon[46738]: Deploying daemon mgr.y on smithi066 2024-05-08T22:30:57.034 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:57.034 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:57.034 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:56 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:56.767+0000 7f8a726a5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:30:57.034 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:56 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:56.877+0000 7f8a726a5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:30:57.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:56 smithi062 ceph-mon[42571]: Deploying daemon mgr.y on smithi066 2024-05-08T22:30:57.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:57.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:57.257 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:56 smithi066 ceph-mon[46830]: Deploying daemon mgr.y on smithi066 2024-05-08T22:30:57.257 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:57.257 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:57.258 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:56 smithi066 ceph-mgr[48330]: -- 172.21.15.66:0/3267224006 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5629955c41a0 con 0x56299559ac00 2024-05-08T22:30:57.258 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:56 smithi066 bash[48292]: ad444d37b03fd9eedb73fe8d23942f63ac61c3eb9a922b66e9cbad25fb88b1c9 2024-05-08T22:30:57.258 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:56 smithi066 systemd[1]: Started Ceph mgr.y for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:30:57.358 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:57.257+0000 7f0a73b5b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T22:30:57.388 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:57 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:57.034+0000 7f8a726a5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:30:57.388 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:57 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:57.173+0000 7f8a726a5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:30:57.388 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:57 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:57.265+0000 7f8a726a5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:30:57.550 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:57.357+0000 7f0a73b5b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T22:30:57.550 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:57.456+0000 7f0a73b5b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T22:30:57.550 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:57.550+0000 7f0a73b5b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T22:30:57.633 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T22:30:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T22:30:57.634 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-05-08T22:30:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-05-08T22:30:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112136 s, 935 MB/s 2024-05-08T22:30:57.634 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 2024-05-08T22:30:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-05-08T22:30:57.925 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:57 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:57.666+0000 7f8a726a5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:30:57.960 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2024-05-08T22:30:58.199 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:30:58.236 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:57 smithi066 ceph-mon[46830]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:58.237 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.237 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.237 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:58.237 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.237 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.237 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.237 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.237 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:58.237 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:58.237 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.268 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:58.073+0000 7f8a726a5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:30:58.268 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:57 smithi079 ceph-mon[46738]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:58.268 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.268 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.268 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:58.268 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.268 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.269 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.269 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.269 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:58.269 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:58.269 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:57 smithi062 ceph-mon[42571]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:30:58.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:30:58.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:58.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:30:58.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:30:58.497 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:58.236+0000 7f0a73b5b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:30:58.638 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:58.268+0000 7f8a726a5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:30:58.638 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:58.361+0000 7f8a726a5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:30:58.823 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:58.497+0000 7f0a73b5b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T22:30:58.823 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:58.586+0000 7f0a73b5b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T22:30:58.824 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:58.699+0000 7f0a73b5b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T22:30:58.992 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:58 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:30:58.992 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:58 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:30:58.992 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:58 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:59.214 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:59 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:59.118+0000 7f8a726a5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:30:59.215 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:58 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:30:59.215 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:58 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:30:59.215 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:58 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:58 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T22:30:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:58 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T22:30:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:58 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:30:59.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:58.863+0000 7f0a73b5b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T22:30:59.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:58.997+0000 7f0a73b5b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T22:30:59.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:59 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:59.086+0000 7f0a73b5b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T22:30:59.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.439+0000 7f691f946700 1 -- 172.21.15.62:0/1326022036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69180fff00 msgr2=0x7f6918100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:59.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.439+0000 7f691f946700 1 --2- 172.21.15.62:0/1326022036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69180fff00 0x7f6918100320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6908009990 tx=0x7f6908009ca0 comp rx=0 tx=0).stop 2024-05-08T22:30:59.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.439+0000 7f691f946700 1 -- 172.21.15.62:0/1326022036 shutdown_connections 2024-05-08T22:30:59.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.439+0000 7f691f946700 1 --2- 172.21.15.62:0/1326022036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6918101a20 0x7f6918108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:59.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.439+0000 7f691f946700 1 --2- 172.21.15.62:0/1326022036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69180fff00 0x7f6918100320 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:59.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.439+0000 7f691f946700 1 --2- 172.21.15.62:0/1326022036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6918101060 0x7f69181014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:59.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.439+0000 7f691f946700 1 -- 172.21.15.62:0/1326022036 >> 172.21.15.62:0/1326022036 conn(0x7f69180fb4c0 msgr2=0x7f69180fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:30:59.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.439+0000 7f691f946700 1 -- 172.21.15.62:0/1326022036 shutdown_connections 2024-05-08T22:30:59.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.439+0000 7f691f946700 1 -- 172.21.15.62:0/1326022036 wait complete. 2024-05-08T22:30:59.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691f946700 1 Processor -- start 2024-05-08T22:30:59.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691f946700 1 -- start start 2024-05-08T22:30:59.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691f946700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69180fff00 0x7f6918196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:59.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691f946700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6918101060 0x7f69181974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:59.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691f946700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6918101a20 0x7f691819e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:59.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691f946700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691819ead0 con 0x7f6918101a20 2024-05-08T22:30:59.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691f946700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691819ec10 con 0x7f6918101060 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691f946700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691819eda0 con 0x7f69180fff00 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691dee3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6918101a20 0x7f691819e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691d6e2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69180fff00 0x7f6918196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691cee1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6918101060 0x7f69181974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691d6e2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69180fff00 0x7f6918196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.62:52222/0 (socket says 172.21.15.62:52222) 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691dee3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6918101a20 0x7f691819e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46574/0 (socket says 172.21.15.62:46574) 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691d6e2700 1 -- 172.21.15.62:0/2562894447 learned_addr learned my addr 172.21.15.62:0/2562894447 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691cee1700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6918101060 0x7f69181974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.62:54058/0 (socket says 172.21.15.62:54058) 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691d6e2700 1 -- 172.21.15.62:0/2562894447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6918101060 msgr2=0x7f69181974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691d6e2700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6918101060 0x7f69181974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691d6e2700 1 -- 172.21.15.62:0/2562894447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6918101a20 msgr2=0x7f691819e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.440+0000 7f691d6e2700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6918101a20 0x7f691819e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:30:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.441+0000 7f691d6e2700 1 -- 172.21.15.62:0/2562894447 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69080095f0 con 0x7f69180fff00 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.441+0000 7f691dee3700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6918101a20 0x7f691819e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.441+0000 7f691d6e2700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69180fff00 0x7f6918196f90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6908005910 tx=0x7f6908004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.441+0000 7f690e7fc700 1 -- 172.21.15.62:0/2562894447 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690801d070 con 0x7f69180fff00 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.441+0000 7f691f946700 1 -- 172.21.15.62:0/2562894447 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691819f020 con 0x7f69180fff00 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.441+0000 7f690e7fc700 1 -- 172.21.15.62:0/2562894447 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6908022470 con 0x7f69180fff00 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.441+0000 7f690e7fc700 1 -- 172.21.15.62:0/2562894447 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908017650 con 0x7f69180fff00 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.441+0000 7f691f946700 1 -- 172.21.15.62:0/2562894447 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691819f490 con 0x7f69180fff00 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.442+0000 7f690e7fc700 1 -- 172.21.15.62:0/2562894447 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 12) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f6908005030 con 0x7f69180fff00 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.442+0000 7f691f946700 1 -- 172.21.15.62:0/2562894447 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691804ec60 con 0x7f69180fff00 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.442+0000 7f690e7fc700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69040389d0 0x7f690403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.442+0000 7f690e7fc700 1 -- 172.21.15.62:0/2562894447 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f690804c5d0 con 0x7f69180fff00 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.442+0000 7f691cee1700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69040389d0 0x7f690403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:30:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.443+0000 7f691cee1700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69040389d0 0x7f690403ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6914009c60 tx=0x7f6914009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:30:59.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.447+0000 7f690e7fc700 1 -- 172.21.15.62:0/2562894447 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6908020370 con 0x7f69180fff00 2024-05-08T22:30:59.638 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:59 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:59.215+0000 7f8a726a5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:30:59.638 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:59 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:59.366+0000 7f8a726a5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:30:59.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:30:59.662+0000 7f691f946700 1 -- 172.21.15.62:0/2562894447 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f69180698d0 con 0x7f69040389d0 2024-05-08T22:30:59.823 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:59 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:59.475+0000 7f0a73b5b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T22:31:00.177 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:30:59 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:30:59.878+0000 7f0a73b5b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T22:31:00.178 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:00 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:00.081+0000 7f0a73b5b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T22:31:00.178 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:00.178 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T22:31:00.178 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: Reconfiguring daemon mgr.x on smithi062 2024-05-08T22:31:00.178 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.178 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.178 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.178 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:00.179 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:00.179 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:00.179 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.179 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:00.179 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:00.179 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:30:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:00.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:00.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T22:31:00.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: Reconfiguring daemon mgr.x on smithi062 2024-05-08T22:31:00.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:00.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:00.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:00.184 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.185 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:00.185 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:00.185 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:30:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:00.370 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:30:59 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:30:59.993+0000 7f8a726a5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:31:00.370 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:31:00 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:31:00.177+0000 7f8a726a5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:31:00.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:00.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T22:31:00.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: Reconfiguring daemon mgr.x on smithi062 2024-05-08T22:31:00.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:00.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:00.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:00.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:00.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:00.372 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:00.372 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:30:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:00.573 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:00 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:00.177+0000 7f0a73b5b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T22:31:00.638 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:31:00 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:31:00.370+0000 7f8a726a5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:31:01.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:00 smithi062 ceph-mon[42571]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:01.241 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:00 smithi066 ceph-mon[46830]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:01.242 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:00 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:00.983+0000 7f0a73b5b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T22:31:01.242 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:01 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:01.084+0000 7f0a73b5b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T22:31:01.297 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:00 smithi079 ceph-mon[46738]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:01.573 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:01 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:01.241+0000 7f0a73b5b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T22:31:01.581 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:31:01 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:31:01.297+0000 7f8a726a5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:31:01.859 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:31:01 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:31:01.581+0000 7f8a726a5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:31:01.859 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:31:01 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:31:01.669+0000 7f8a726a5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:31:02.138 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:31:01 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z[48603]: 2024-05-08T22:31:01.860+0000 7f8a726a5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:31:02.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:01 smithi079 ceph-mon[46738]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:01 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/411742887' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7242466d-5961-4fd3-8d3e-2996fcccd92e"}]: dispatch 2024-05-08T22:31:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:01 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/411742887' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7242466d-5961-4fd3-8d3e-2996fcccd92e"}]': finished 2024-05-08T22:31:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:01 smithi079 ceph-mon[46738]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T22:31:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:01 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:01 smithi079 ceph-mon[46738]: Standby manager daemon z started 2024-05-08T22:31:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:01 smithi079 ceph-mon[46738]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:31:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:01 smithi079 ceph-mon[46738]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:31:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:01 smithi079 ceph-mon[46738]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:31:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:01 smithi079 ceph-mon[46738]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:31:02.290 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:01 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:01.899+0000 7f0a73b5b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T22:31:02.290 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:02 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:02.087+0000 7f0a73b5b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T22:31:02.291 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:01 smithi066 ceph-mon[46830]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:02.291 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:01 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/411742887' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7242466d-5961-4fd3-8d3e-2996fcccd92e"}]: dispatch 2024-05-08T22:31:02.291 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:01 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/411742887' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7242466d-5961-4fd3-8d3e-2996fcccd92e"}]': finished 2024-05-08T22:31:02.291 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:01 smithi066 ceph-mon[46830]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T22:31:02.291 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:01 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:02.291 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:01 smithi066 ceph-mon[46830]: Standby manager daemon z started 2024-05-08T22:31:02.291 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:01 smithi066 ceph-mon[46830]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:31:02.291 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:01 smithi066 ceph-mon[46830]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:31:02.291 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:01 smithi066 ceph-mon[46830]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:31:02.291 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:01 smithi066 ceph-mon[46830]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:31:02.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:01 smithi062 ceph-mon[42571]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:02.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:01 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/411742887' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7242466d-5961-4fd3-8d3e-2996fcccd92e"}]: dispatch 2024-05-08T22:31:02.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:01 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/411742887' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7242466d-5961-4fd3-8d3e-2996fcccd92e"}]': finished 2024-05-08T22:31:02.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:01 smithi062 ceph-mon[42571]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T22:31:02.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:01 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:02.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:01 smithi062 ceph-mon[42571]: Standby manager daemon z started 2024-05-08T22:31:02.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:01 smithi062 ceph-mon[42571]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T22:31:02.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:01 smithi062 ceph-mon[42571]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:31:02.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:01 smithi062 ceph-mon[42571]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T22:31:02.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:01 smithi062 ceph-mon[42571]: from='mgr.? 172.21.15.79:0/2535237393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:31:02.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:02.559+0000 7f690e7fc700 1 -- 172.21.15.62:0/2562894447 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mgrmap(e 13) v1 ==== 90707+0+0 (secure 0 0 0) 0x7f6908017cd0 con 0x7f69180fff00 2024-05-08T22:31:03.264 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:02 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:02.290+0000 7f0a73b5b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T22:31:03.265 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:02 smithi066 ceph-mon[46830]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:03.265 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:02 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1590779465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:03.265 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:02 smithi066 ceph-mon[46830]: mgrmap e13: x(active, since 58s), standbys: z 2024-05-08T22:31:03.265 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:02 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:31:03.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:02 smithi079 ceph-mon[46738]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:03.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:02 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1590779465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:03.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:02 smithi079 ceph-mon[46738]: mgrmap e13: x(active, since 58s), standbys: z 2024-05-08T22:31:03.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:02 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:31:03.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:02 smithi062 ceph-mon[42571]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:03.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:02 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1590779465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:03.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:02 smithi062 ceph-mon[42571]: mgrmap e13: x(active, since 58s), standbys: z 2024-05-08T22:31:03.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:02 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T22:31:03.559 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:03 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:03.264+0000 7f0a73b5b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T22:31:03.823 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:03 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:03.559+0000 7f0a73b5b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T22:31:03.824 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:03 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:03.651+0000 7f0a73b5b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T22:31:03.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:03.955+0000 7f690e7fc700 1 -- 172.21.15.62:0/2562894447 <== mon.1 v2:172.21.15.79:3300/0 8 ==== mgrmap(e 14) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f69080163a0 con 0x7f69180fff00 2024-05-08T22:31:04.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:31:03 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y[48307]: 2024-05-08T22:31:03.848+0000 7f0a73b5b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T22:31:04.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:03 smithi066 ceph-mon[46830]: Standby manager daemon y started 2024-05-08T22:31:04.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:03 smithi066 ceph-mon[46830]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:31:04.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:03 smithi066 ceph-mon[46830]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:31:04.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:03 smithi066 ceph-mon[46830]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:31:04.325 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:03 smithi066 ceph-mon[46830]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:31:04.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:03 smithi079 ceph-mon[46738]: Standby manager daemon y started 2024-05-08T22:31:04.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:03 smithi079 ceph-mon[46738]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:31:04.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:03 smithi079 ceph-mon[46738]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:31:04.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:03 smithi079 ceph-mon[46738]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:31:04.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:03 smithi079 ceph-mon[46738]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:31:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:03 smithi062 ceph-mon[42571]: Standby manager daemon y started 2024-05-08T22:31:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:03 smithi062 ceph-mon[42571]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T22:31:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:03 smithi062 ceph-mon[42571]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T22:31:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:03 smithi062 ceph-mon[42571]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T22:31:04.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:03 smithi062 ceph-mon[42571]: from='mgr.? 172.21.15.66:0/1486347733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T22:31:05.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:04 smithi066 ceph-mon[46830]: mgrmap e14: x(active, since 60s), standbys: z, y 2024-05-08T22:31:05.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:31:05.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:04 smithi066 ceph-mon[46830]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:05.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:04 smithi079 ceph-mon[46738]: mgrmap e14: x(active, since 60s), standbys: z, y 2024-05-08T22:31:05.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:31:05.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:04 smithi079 ceph-mon[46738]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:05.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:04 smithi062 ceph-mon[42571]: mgrmap e14: x(active, since 60s), standbys: z, y 2024-05-08T22:31:05.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T22:31:05.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:04 smithi062 ceph-mon[42571]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:07.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:07 smithi062 ceph-mon[42571]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:07.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:07 smithi066 ceph-mon[46830]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:07.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:07 smithi079 ceph-mon[46738]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:08.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:08 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T22:31:08.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:08 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:08.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:08 smithi066 ceph-mon[46830]: Deploying daemon osd.0 on smithi062 2024-05-08T22:31:08.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:08 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T22:31:08.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:08 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:08.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:08 smithi079 ceph-mon[46738]: Deploying daemon osd.0 on smithi062 2024-05-08T22:31:08.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:08 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T22:31:08.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:08 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:08.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:08 smithi062 ceph-mon[42571]: Deploying daemon osd.0 on smithi062 2024-05-08T22:31:09.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:09 smithi062 ceph-mon[42571]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:09.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:09 smithi066 ceph-mon[46830]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:09.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:09 smithi079 ceph-mon[46738]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:11.278 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:11 smithi062 ceph-mon[42571]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:11.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:11 smithi066 ceph-mon[46830]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:11.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:11 smithi079 ceph-mon[46738]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:12.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:12 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:12.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:12 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:12.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:12 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:12.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:12 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:12.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:12 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:12.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:12 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:12.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:12 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:12.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:12 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:12.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:12 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:13.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.663+0000 7f690e7fc700 1 -- 172.21.15.62:0/2562894447 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f69180698d0 con 0x7f69040389d0 2024-05-08T22:31:13.663 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 0 on host 'smithi062' 2024-05-08T22:31:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.665+0000 7f691f946700 1 -- 172.21.15.62:0/2562894447 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69040389d0 msgr2=0x7f690403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.665+0000 7f691f946700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69040389d0 0x7f690403ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6914009c60 tx=0x7f6914009260 comp rx=0 tx=0).stop 2024-05-08T22:31:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.665+0000 7f691f946700 1 -- 172.21.15.62:0/2562894447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69180fff00 msgr2=0x7f6918196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.665+0000 7f691f946700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69180fff00 0x7f6918196f90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6908005910 tx=0x7f6908004730 comp rx=0 tx=0).stop 2024-05-08T22:31:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.665+0000 7f691f946700 1 -- 172.21.15.62:0/2562894447 shutdown_connections 2024-05-08T22:31:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.665+0000 7f691f946700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6918101a20 0x7f691819e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.665+0000 7f691f946700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69180fff00 0x7f6918196f90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.665+0000 7f691f946700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69040389d0 0x7f690403ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.665+0000 7f691f946700 1 --2- 172.21.15.62:0/2562894447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6918101060 0x7f69181974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.665+0000 7f691f946700 1 -- 172.21.15.62:0/2562894447 >> 172.21.15.62:0/2562894447 conn(0x7f69180fb4c0 msgr2=0x7f69180691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:31:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.666+0000 7f691f946700 1 -- 172.21.15.62:0/2562894447 shutdown_connections 2024-05-08T22:31:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:13.666+0000 7f691f946700 1 -- 172.21.15.62:0/2562894447 wait complete. 2024-05-08T22:31:13.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:13 smithi062 ceph-mon[42571]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:13.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:13 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:13.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:13 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:13.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:13 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:13.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:13 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:13.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:13 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:13.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:13 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:13.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:13 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:13.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:13 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:14.046 DEBUG:teuthology.orchestra.run.smithi062:osd.0> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.0.service 2024-05-08T22:31:14.048 INFO:tasks.cephadm:Deploying osd.1 on smithi066 with /dev/vg_nvme/lv_4... 2024-05-08T22:31:14.048 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T22:31:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:13 smithi066 ceph-mon[46830]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:13 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:13 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:13 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:13 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:13 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:13 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:13 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:13 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:14.080 INFO:journalctl@ceph.osd.0.smithi062.stdout:-- Logs begin at Wed 2024-05-08 22:12:07 UTC. -- 2024-05-08T22:31:14.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:13 smithi079 ceph-mon[46738]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:14.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:13 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:13 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:13 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:13 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:13 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:13 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:13 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:13 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:14.287 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:31:15.568 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T22:31:15.568 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T22:31:15.568 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-05-08T22:31:15.569 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-05-08T22:31:15.569 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128085 s, 819 MB/s 2024-05-08T22:31:15.569 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-05-08T22:31:15.672 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:31:15 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-0[61728]: 2024-05-08T22:31:15.309+0000 7ff0b457f7c0 -1 osd.0 0 log_to_monitors true 2024-05-08T22:31:15.896 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:15 smithi066 ceph-mon[46830]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:15.896 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:15 smithi066 ceph-mon[46830]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:31:15.897 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_4 2024-05-08T22:31:16.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:15 smithi062 ceph-mon[42571]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:16.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:15 smithi062 ceph-mon[42571]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:31:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:15 smithi079 ceph-mon[46738]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:15 smithi079 ceph-mon[46738]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T22:31:16.161 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:31:17.015 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:16 smithi066 ceph-mon[46830]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T22:31:17.015 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:16 smithi066 ceph-mon[46830]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T22:31:17.015 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:16 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:17.015 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:16 smithi066 ceph-mon[46830]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:31:17.015 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:16 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.015 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:16 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.015 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:16 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:31:17.015 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:16 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.016 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:16 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:17.016 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:16 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:17.016 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:16 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:16 smithi079 ceph-mon[46738]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T22:31:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:16 smithi079 ceph-mon[46738]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T22:31:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:16 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:16 smithi079 ceph-mon[46738]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:31:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:16 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:16 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:16 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:31:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:16 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:16 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:16 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:16 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.171 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:31:16 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-0[61728]: 2024-05-08T22:31:16.696+0000 7ff0a93e2700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T22:31:17.171 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:31:16 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-0[61728]: 2024-05-08T22:31:16.702+0000 7ff0a59d8700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:31:17.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:16 smithi062 ceph-mon[42571]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T22:31:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:16 smithi062 ceph-mon[42571]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T22:31:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:16 smithi062 ceph-mon[42571]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:31:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:31:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:17.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.203+0000 7f835ae60700 1 -- 172.21.15.66:0/2736193116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83541021e0 msgr2=0x7f8354102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:17.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.203+0000 7f835ae60700 1 --2- 172.21.15.66:0/2736193116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83541021e0 0x7f8354102640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f83400099c0 tx=0x7f8340009cd0 comp rx=0 tx=0).stop 2024-05-08T22:31:17.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.203+0000 7f835ae60700 1 -- 172.21.15.66:0/2736193116 shutdown_connections 2024-05-08T22:31:17.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.203+0000 7f835ae60700 1 --2- 172.21.15.66:0/2736193116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8354102b80 0x7f8354109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:17.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.203+0000 7f835ae60700 1 --2- 172.21.15.66:0/2736193116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8354100fe0 0x7f8354101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:17.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.203+0000 7f835ae60700 1 --2- 172.21.15.66:0/2736193116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83541021e0 0x7f8354102640 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:17.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.203+0000 7f835ae60700 1 -- 172.21.15.66:0/2736193116 >> 172.21.15.66:0/2736193116 conn(0x7f83540fc580 msgr2=0x7f83540fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:31:17.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.203+0000 7f835ae60700 1 -- 172.21.15.66:0/2736193116 shutdown_connections 2024-05-08T22:31:17.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.203+0000 7f835ae60700 1 -- 172.21.15.66:0/2736193116 wait complete. 2024-05-08T22:31:17.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f835ae60700 1 Processor -- start 2024-05-08T22:31:17.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f835ae60700 1 -- start start 2024-05-08T22:31:17.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f835ae60700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354100fe0 0x7f8354196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:17.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f835ae60700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83541021e0 0x7f8354197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f835ae60700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8354102b80 0x7f835419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f835ae60700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835419e980 con 0x7f83541021e0 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f835ae60700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835419eac0 con 0x7f8354100fe0 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f835ae60700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835419ec50 con 0x7f8354102b80 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f8358bfc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354100fe0 0x7f8354196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f8358bfc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354100fe0 0x7f8354196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46546/0 (socket says 172.21.15.66:46546) 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f8358bfc700 1 -- 172.21.15.66:0/959836508 learned_addr learned my addr 172.21.15.66:0/959836508 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f8353fff700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83541021e0 0x7f8354197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f83593fd700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8354102b80 0x7f835419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f8358bfc700 1 -- 172.21.15.66:0/959836508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8354102b80 msgr2=0x7f835419e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f8358bfc700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8354102b80 0x7f835419e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f8358bfc700 1 -- 172.21.15.66:0/959836508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83541021e0 msgr2=0x7f8354197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:17.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f8358bfc700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83541021e0 0x7f8354197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:17.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f8358bfc700 1 -- 172.21.15.66:0/959836508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400095f0 con 0x7f8354100fe0 2024-05-08T22:31:17.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f8353fff700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83541021e0 0x7f8354197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:31:17.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.204+0000 7f8358bfc700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354100fe0 0x7f8354196e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f834800d980 tx=0x7f834800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:31:17.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.205+0000 7f8351ffb700 1 -- 172.21.15.66:0/959836508 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800de60 con 0x7f8354100fe0 2024-05-08T22:31:17.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.205+0000 7f8351ffb700 1 -- 172.21.15.66:0/959836508 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83480044e0 con 0x7f8354100fe0 2024-05-08T22:31:17.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.205+0000 7f835ae60700 1 -- 172.21.15.66:0/959836508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835419ef30 con 0x7f8354100fe0 2024-05-08T22:31:17.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.205+0000 7f8351ffb700 1 -- 172.21.15.66:0/959836508 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348003bc0 con 0x7f8354100fe0 2024-05-08T22:31:17.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.205+0000 7f835ae60700 1 -- 172.21.15.66:0/959836508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835419f350 con 0x7f8354100fe0 2024-05-08T22:31:17.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.206+0000 7f835ae60700 1 -- 172.21.15.66:0/959836508 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83541083c0 con 0x7f8354100fe0 2024-05-08T22:31:17.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.207+0000 7f8351ffb700 1 -- 172.21.15.66:0/959836508 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 14) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f8348009bb0 con 0x7f8354100fe0 2024-05-08T22:31:17.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.208+0000 7f8351ffb700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f83440a1190 0x7f83440a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:17.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.208+0000 7f8351ffb700 1 -- 172.21.15.66:0/959836508 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7f83480caa40 con 0x7f8354100fe0 2024-05-08T22:31:17.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.208+0000 7f8353fff700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f83440a1190 0x7f83440a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:17.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.208+0000 7f8353fff700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f83440a1190 0x7f83440a3650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8340000c00 tx=0x7f8340005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:31:17.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.212+0000 7f8351ffb700 1 -- 172.21.15.66:0/959836508 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8348095cc0 con 0x7f8354100fe0 2024-05-08T22:31:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:17.424+0000 7f835ae60700 1 -- 172.21.15.66:0/959836508 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f835402ce60 con 0x7f83440a1190 2024-05-08T22:31:17.964 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:17 smithi066 ceph-mon[46830]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:17.964 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:17 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi062 2024-05-08T22:31:17.964 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:17 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi062 to 17114M 2024-05-08T22:31:17.964 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:17 smithi066 ceph-mon[46830]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:31:17.964 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:17 smithi066 ceph-mon[46830]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T22:31:17.965 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:17.965 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:17.965 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:17.965 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:17.965 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:17.966 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:17 smithi079 ceph-mon[46738]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:17 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi062 2024-05-08T22:31:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:17 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi062 to 17114M 2024-05-08T22:31:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:17 smithi079 ceph-mon[46738]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:31:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:17 smithi079 ceph-mon[46738]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T22:31:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:17 smithi062 ceph-mon[42571]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:17 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi062 2024-05-08T22:31:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:17 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi062 to 17114M 2024-05-08T22:31:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:17 smithi062 ceph-mon[42571]: from='osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:31:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:17 smithi062 ceph-mon[42571]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T22:31:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:19.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:18 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:31:19.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:18 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:31:19.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:18 smithi066 ceph-mon[46830]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:19.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:18 smithi066 ceph-mon[46830]: osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645] boot 2024-05-08T22:31:19.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:18 smithi066 ceph-mon[46830]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T22:31:19.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:18 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:19.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:18 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:31:19.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:18 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:31:19.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:18 smithi079 ceph-mon[46738]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:19.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:18 smithi079 ceph-mon[46738]: osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645] boot 2024-05-08T22:31:19.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:18 smithi079 ceph-mon[46738]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T22:31:19.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:18 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:19.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:18 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:31:19.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:18 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:31:19.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:18 smithi062 ceph-mon[42571]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:19.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:18 smithi062 ceph-mon[42571]: osd.0 [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645] boot 2024-05-08T22:31:19.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:18 smithi062 ceph-mon[42571]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T22:31:19.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:18 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T22:31:20.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:19 smithi066 ceph-mon[46830]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:20.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:19 smithi066 ceph-mon[46830]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T22:31:20.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:19 smithi066 ceph-mon[46830]: from='client.? 172.21.15.66:0/957702713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e358e76-c55b-4e94-a8e6-727c65c10718"}]: dispatch 2024-05-08T22:31:20.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:19 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e358e76-c55b-4e94-a8e6-727c65c10718"}]: dispatch 2024-05-08T22:31:20.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:19 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e358e76-c55b-4e94-a8e6-727c65c10718"}]': finished 2024-05-08T22:31:20.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:19 smithi066 ceph-mon[46830]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T22:31:20.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:19 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:19 smithi079 ceph-mon[46738]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:19 smithi079 ceph-mon[46738]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T22:31:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:19 smithi079 ceph-mon[46738]: from='client.? 172.21.15.66:0/957702713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e358e76-c55b-4e94-a8e6-727c65c10718"}]: dispatch 2024-05-08T22:31:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:19 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e358e76-c55b-4e94-a8e6-727c65c10718"}]: dispatch 2024-05-08T22:31:20.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:19 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e358e76-c55b-4e94-a8e6-727c65c10718"}]': finished 2024-05-08T22:31:20.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:19 smithi079 ceph-mon[46738]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T22:31:20.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:19 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:20.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:19 smithi062 ceph-mon[42571]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T22:31:20.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:19 smithi062 ceph-mon[42571]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T22:31:20.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:19 smithi062 ceph-mon[42571]: from='client.? 172.21.15.66:0/957702713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e358e76-c55b-4e94-a8e6-727c65c10718"}]: dispatch 2024-05-08T22:31:20.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:19 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e358e76-c55b-4e94-a8e6-727c65c10718"}]: dispatch 2024-05-08T22:31:20.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:19 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e358e76-c55b-4e94-a8e6-727c65c10718"}]': finished 2024-05-08T22:31:20.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:19 smithi062 ceph-mon[42571]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T22:31:20.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:19 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:21.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:20 smithi066 ceph-mon[46830]: from='client.? 172.21.15.66:0/26667066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:21.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:20 smithi079 ceph-mon[46738]: from='client.? 172.21.15.66:0/26667066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:21.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:20 smithi062 ceph-mon[42571]: from='client.? 172.21.15.66:0/26667066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:22.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:21 smithi066 ceph-mon[46830]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:22.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:21 smithi079 ceph-mon[46738]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:22.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:21 smithi062 ceph-mon[42571]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:23.995 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:23 smithi066 ceph-mon[46830]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:24.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:23 smithi079 ceph-mon[46738]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:24.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:23 smithi062 ceph-mon[42571]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:26.004 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:25 smithi066 ceph-mon[46830]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:26.005 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:25 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T22:31:26.005 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:25 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:26.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:25 smithi079 ceph-mon[46738]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:26.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:25 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T22:31:26.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:25 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:26.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:25 smithi062 ceph-mon[42571]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:26.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:25 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T22:31:26.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:25 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:27.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:26 smithi066 ceph-mon[46830]: Deploying daemon osd.1 on smithi066 2024-05-08T22:31:27.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:26 smithi079 ceph-mon[46738]: Deploying daemon osd.1 on smithi066 2024-05-08T22:31:27.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:26 smithi062 ceph-mon[42571]: Deploying daemon osd.1 on smithi066 2024-05-08T22:31:27.924 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:27 smithi066 ceph-mon[46830]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:28.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:27 smithi079 ceph-mon[46738]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:28.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:27 smithi062 ceph-mon[42571]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:30.038 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:29 smithi066 ceph-mon[46830]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:30.038 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:30.039 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:30.039 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:30.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:29 smithi079 ceph-mon[46738]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:30.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:30.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:30.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:30.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:29 smithi062 ceph-mon[42571]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:30.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:30.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:30.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.532+0000 7f8351ffb700 1 -- 172.21.15.66:0/959836508 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f835402ce60 con 0x7f83440a1190 2024-05-08T22:31:31.532 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 1 on host 'smithi066' 2024-05-08T22:31:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 -- 172.21.15.66:0/959836508 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f83440a1190 msgr2=0x7f83440a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f83440a1190 0x7f83440a3650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8340000c00 tx=0x7f8340005fd0 comp rx=0 tx=0).stop 2024-05-08T22:31:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 -- 172.21.15.66:0/959836508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354100fe0 msgr2=0x7f8354196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354100fe0 0x7f8354196e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f834800d980 tx=0x7f834800dd40 comp rx=0 tx=0).stop 2024-05-08T22:31:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 -- 172.21.15.66:0/959836508 shutdown_connections 2024-05-08T22:31:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8354102b80 0x7f835419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:31.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f83440a1190 0x7f83440a3650 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:31.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354100fe0 0x7f8354196e40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:31.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 --2- 172.21.15.66:0/959836508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83541021e0 0x7f8354197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:31.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 -- 172.21.15.66:0/959836508 >> 172.21.15.66:0/959836508 conn(0x7f83540fc580 msgr2=0x7f8354105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:31:31.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 -- 172.21.15.66:0/959836508 shutdown_connections 2024-05-08T22:31:31.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:31:31.534+0000 7f835ae60700 1 -- 172.21.15.66:0/959836508 wait complete. 2024-05-08T22:31:31.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:31 smithi066 ceph-mon[46830]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:31.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:31 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:31 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:31 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:31.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:31 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:31.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:31 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:31 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:31 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:31 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:31.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:31 smithi079 ceph-mon[46738]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:31.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:31 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:31 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:31 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:31.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:31 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:31.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:31 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:31 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:31 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:31 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:31.894 DEBUG:teuthology.orchestra.run.smithi066:osd.1> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.1.service 2024-05-08T22:31:31.897 INFO:tasks.cephadm:Deploying osd.2 on smithi079 with /dev/vg_nvme/lv_4... 2024-05-08T22:31:31.897 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T22:31:31.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:31 smithi062 ceph-mon[42571]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:31.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:31.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:31.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:31.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:31.928 INFO:journalctl@ceph.osd.1.smithi066.stdout:-- Logs begin at Wed 2024-05-08 22:13:17 UTC. -- 2024-05-08T22:31:32.122 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:31:33.406 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T22:31:33.406 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T22:31:33.407 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-05-08T22:31:33.407 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-05-08T22:31:33.407 INFO:teuthology.orchestra.run.smithi079.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0156721 s, 669 MB/s 2024-05-08T22:31:33.407 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-05-08T22:31:33.532 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:31:33 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-1[51799]: 2024-05-08T22:31:33.106+0000 7ff0b98ba7c0 -1 osd.1 0 log_to_monitors true 2024-05-08T22:31:33.749 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:33 smithi079 ceph-mon[46738]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:33.750 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:33 smithi079 ceph-mon[46738]: from='osd.1 [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:31:33.750 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:33 smithi079 ceph-mon[46738]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:31:33.768 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_4 2024-05-08T22:31:33.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:33 smithi066 ceph-mon[46830]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:33.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:33 smithi066 ceph-mon[46830]: from='osd.1 [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:31:33.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:33 smithi066 ceph-mon[46830]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:31:33.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:33 smithi062 ceph-mon[42571]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:33.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:33 smithi062 ceph-mon[42571]: from='osd.1 [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:31:33.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:33 smithi062 ceph-mon[42571]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T22:31:33.984 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:31:34.813 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T22:31:34.813 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: from='osd.1 [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi066 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi066 to 17113M 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:34.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:34.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: from='osd.1 [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi066 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi066 to 17113M 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:34.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:34 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.993+0000 7f584a959700 1 -- 172.21.15.79:0/4066992496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5844102b80 msgr2=0x7f5844109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.993+0000 7f584a959700 1 --2- 172.21.15.79:0/4066992496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5844102b80 0x7f5844109430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f583c00b0a0 tx=0x7f583c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:31:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.993+0000 7f584a959700 1 -- 172.21.15.79:0/4066992496 shutdown_connections 2024-05-08T22:31:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.993+0000 7f584a959700 1 --2- 172.21.15.79:0/4066992496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5844102b80 0x7f5844109430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.993+0000 7f584a959700 1 --2- 172.21.15.79:0/4066992496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5844100fe0 0x7f5844101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.993+0000 7f584a959700 1 --2- 172.21.15.79:0/4066992496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58441021e0 0x7f5844102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.993+0000 7f584a959700 1 -- 172.21.15.79:0/4066992496 >> 172.21.15.79:0/4066992496 conn(0x7f58440fc580 msgr2=0x7f58440fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:31:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.993+0000 7f584a959700 1 -- 172.21.15.79:0/4066992496 shutdown_connections 2024-05-08T22:31:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.993+0000 7f584a959700 1 -- 172.21.15.79:0/4066992496 wait complete. 2024-05-08T22:31:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.993+0000 7f584a959700 1 Processor -- start 2024-05-08T22:31:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.993+0000 7f584a959700 1 -- start start 2024-05-08T22:31:34.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f584a959700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5844100fe0 0x7f5844196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:34.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f584a959700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58441021e0 0x7f5844197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:34.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f584a959700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5844102b80 0x7f584419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:34.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f584a959700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584419eb10 con 0x7f5844102b80 2024-05-08T22:31:34.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f584a959700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584419ec50 con 0x7f58441021e0 2024-05-08T22:31:34.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f584a959700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584419ede0 con 0x7f5844100fe0 2024-05-08T22:31:34.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f5843fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5844100fe0 0x7f5844196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f5843fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5844100fe0 0x7f5844196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57980/0 (socket says 172.21.15.79:57980) 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f5843fff700 1 -- 172.21.15.79:0/510238936 learned_addr learned my addr 172.21.15.79:0/510238936 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f58437fe700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58441021e0 0x7f5844197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f5848ef6700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5844102b80 0x7f584419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f5843fff700 1 -- 172.21.15.79:0/510238936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58441021e0 msgr2=0x7f5844197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f5843fff700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58441021e0 0x7f5844197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f5843fff700 1 -- 172.21.15.79:0/510238936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5844102b80 msgr2=0x7f584419e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f5843fff700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5844102b80 0x7f584419e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f5843fff700 1 -- 172.21.15.79:0/510238936 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c009d00 con 0x7f5844100fe0 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f5843fff700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5844100fe0 0x7f5844196fd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f583400e980 tx=0x7f583400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f58417fa700 1 -- 172.21.15.79:0/510238936 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834009770 con 0x7f5844100fe0 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f58417fa700 1 -- 172.21.15.79:0/510238936 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f583400eed0 con 0x7f5844100fe0 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f584a959700 1 -- 172.21.15.79:0/510238936 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584419f0c0 con 0x7f5844100fe0 2024-05-08T22:31:34.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f58417fa700 1 -- 172.21.15.79:0/510238936 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834003cd0 con 0x7f5844100fe0 2024-05-08T22:31:34.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.994+0000 7f584a959700 1 -- 172.21.15.79:0/510238936 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584419f570 con 0x7f5844100fe0 2024-05-08T22:31:34.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.995+0000 7f584a959700 1 -- 172.21.15.79:0/510238936 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5844108470 con 0x7f5844100fe0 2024-05-08T22:31:34.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.996+0000 7f58417fa700 1 -- 172.21.15.79:0/510238936 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 14) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f583400cbd0 con 0x7f5844100fe0 2024-05-08T22:31:34.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.997+0000 7f58417fa700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f582c0a1090 0x7f582c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:34.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.997+0000 7f58417fa700 1 -- 172.21.15.79:0/510238936 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7f5834014070 con 0x7f5844100fe0 2024-05-08T22:31:34.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.997+0000 7f58437fe700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f582c0a1090 0x7f582c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:34.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:34.997+0000 7f58437fe700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f582c0a1090 0x7f582c0a3550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5838005950 tx=0x7f583800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:31:35.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:35.001+0000 7f58417fa700 1 -- 172.21.15.79:0/510238936 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f58340962e0 con 0x7f5844100fe0 2024-05-08T22:31:35.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T22:31:35.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T22:31:35.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:35.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: from='osd.1 [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:31:35.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:31:35.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi066 2024-05-08T22:31:35.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:35.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:35.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:31:35.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi066 to 17113M 2024-05-08T22:31:35.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:35.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:35.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:35.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:34 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:35.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:35.202+0000 7f584a959700 1 -- 172.21.15.79:0/510238936 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f58440613e0 con 0x7f582c0a1090 2024-05-08T22:31:35.823 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:31:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-1[51799]: 2024-05-08T22:31:35.474+0000 7ff0aff20700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T22:31:35.824 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:31:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-1[51799]: 2024-05-08T22:31:35.481+0000 7ff0a6d0b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:31:35.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:35 smithi066 ceph-mon[46830]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:35.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:35 smithi066 ceph-mon[46830]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:31:35.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:35 smithi066 ceph-mon[46830]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T22:31:35.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:35.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:35.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:35.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:35.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:35.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:35 smithi066 ceph-mon[46830]: from='osd.1 ' entity='osd.1' 2024-05-08T22:31:35.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:35 smithi079 ceph-mon[46738]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:35.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:35 smithi079 ceph-mon[46738]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:31:35.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:35 smithi079 ceph-mon[46738]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T22:31:35.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:35.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:35.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:35.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:35.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:35.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:35 smithi079 ceph-mon[46738]: from='osd.1 ' entity='osd.1' 2024-05-08T22:31:35.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:35 smithi062 ceph-mon[42571]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:35 smithi062 ceph-mon[42571]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:31:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:35 smithi062 ceph-mon[42571]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T22:31:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:35 smithi062 ceph-mon[42571]: from='osd.1 ' entity='osd.1' 2024-05-08T22:31:36.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:36 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:31:36.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:36 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:31:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:36 smithi062 ceph-mon[42571]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:36 smithi062 ceph-mon[42571]: osd.1 [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054] boot 2024-05-08T22:31:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:36 smithi062 ceph-mon[42571]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T22:31:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:37.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:36 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:31:37.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:36 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:31:37.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:36 smithi066 ceph-mon[46830]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:36 smithi066 ceph-mon[46830]: osd.1 [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054] boot 2024-05-08T22:31:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:36 smithi066 ceph-mon[46830]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T22:31:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:37.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:36 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:31:37.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:36 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:31:37.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:36 smithi079 ceph-mon[46738]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:37.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:36 smithi079 ceph-mon[46738]: osd.1 [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054] boot 2024-05-08T22:31:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:36 smithi079 ceph-mon[46738]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T22:31:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T22:31:37.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:37 smithi062 ceph-mon[42571]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:37.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:37 smithi062 ceph-mon[42571]: from='client.? 172.21.15.79:0/1343454866' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8002d5-c2cc-4275-aa59-bb335b2b8740"}]: dispatch 2024-05-08T22:31:37.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:37 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8002d5-c2cc-4275-aa59-bb335b2b8740"}]: dispatch 2024-05-08T22:31:37.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:37 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a8002d5-c2cc-4275-aa59-bb335b2b8740"}]': finished 2024-05-08T22:31:37.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:37 smithi062 ceph-mon[42571]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T22:31:37.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:37 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:38.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:37 smithi066 ceph-mon[46830]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:38.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:37 smithi066 ceph-mon[46830]: from='client.? 172.21.15.79:0/1343454866' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8002d5-c2cc-4275-aa59-bb335b2b8740"}]: dispatch 2024-05-08T22:31:38.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:37 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8002d5-c2cc-4275-aa59-bb335b2b8740"}]: dispatch 2024-05-08T22:31:38.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:37 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a8002d5-c2cc-4275-aa59-bb335b2b8740"}]': finished 2024-05-08T22:31:38.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:37 smithi066 ceph-mon[46830]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T22:31:38.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:37 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:38.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:37 smithi079 ceph-mon[46738]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T22:31:38.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:37 smithi079 ceph-mon[46738]: from='client.? 172.21.15.79:0/1343454866' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8002d5-c2cc-4275-aa59-bb335b2b8740"}]: dispatch 2024-05-08T22:31:38.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:37 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8002d5-c2cc-4275-aa59-bb335b2b8740"}]: dispatch 2024-05-08T22:31:38.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:37 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a8002d5-c2cc-4275-aa59-bb335b2b8740"}]': finished 2024-05-08T22:31:38.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:37 smithi079 ceph-mon[46738]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T22:31:38.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:37 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:38.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:38 smithi062 ceph-mon[42571]: from='client.? 172.21.15.79:0/4071696319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:38.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:38 smithi062 ceph-mon[42571]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T22:31:38.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:38 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:39.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:38 smithi066 ceph-mon[46830]: from='client.? 172.21.15.79:0/4071696319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:39.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:38 smithi066 ceph-mon[46830]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T22:31:39.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:38 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:39.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:38 smithi079 ceph-mon[46738]: from='client.? 172.21.15.79:0/4071696319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:39.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:38 smithi079 ceph-mon[46738]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T22:31:39.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:38 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:39.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:39 smithi062 ceph-mon[42571]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:40.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:39 smithi066 ceph-mon[46830]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:40.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:39 smithi079 ceph-mon[46738]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:41.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:41 smithi062 ceph-mon[42571]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:42.036 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:41 smithi079 ceph-mon[46738]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:42.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:41 smithi066 ceph-mon[46830]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:43.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:43 smithi079 ceph-mon[46738]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:43.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T22:31:43.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:43.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:43 smithi062 ceph-mon[42571]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:43.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T22:31:43.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:44.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:43 smithi066 ceph-mon[46830]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:44.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T22:31:44.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:44.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:44 smithi079 ceph-mon[46738]: Deploying daemon osd.2 on smithi079 2024-05-08T22:31:44.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:44 smithi062 ceph-mon[42571]: Deploying daemon osd.2 on smithi079 2024-05-08T22:31:45.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:44 smithi066 ceph-mon[46830]: Deploying daemon osd.2 on smithi079 2024-05-08T22:31:45.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:45 smithi062 ceph-mon[42571]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:46.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:45 smithi066 ceph-mon[46830]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:46.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:45 smithi079 ceph-mon[46738]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:48.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:47 smithi079 ceph-mon[46738]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:48.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:48.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:47 smithi062 ceph-mon[42571]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:48.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:48.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:48.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:48.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:47 smithi066 ceph-mon[46830]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:48.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:48.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:48.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:49.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.157+0000 7f58417fa700 1 -- 172.21.15.79:0/510238936 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f58440613e0 con 0x7f582c0a1090 2024-05-08T22:31:49.157 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 2 on host 'smithi079' 2024-05-08T22:31:49.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 -- 172.21.15.79:0/510238936 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f582c0a1090 msgr2=0x7f582c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:49.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f582c0a1090 0x7f582c0a3550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5838005950 tx=0x7f583800b410 comp rx=0 tx=0).stop 2024-05-08T22:31:49.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 -- 172.21.15.79:0/510238936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5844100fe0 msgr2=0x7f5844196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:49.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5844100fe0 0x7f5844196fd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f583400e980 tx=0x7f583400ec90 comp rx=0 tx=0).stop 2024-05-08T22:31:49.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 -- 172.21.15.79:0/510238936 shutdown_connections 2024-05-08T22:31:49.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5844102b80 0x7f584419e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:49.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5844100fe0 0x7f5844196fd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:49.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f582c0a1090 0x7f582c0a3550 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:49.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 --2- 172.21.15.79:0/510238936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58441021e0 0x7f5844197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:49.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 -- 172.21.15.79:0/510238936 >> 172.21.15.79:0/510238936 conn(0x7f58440fc580 msgr2=0x7f58441055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:31:49.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 -- 172.21.15.79:0/510238936 shutdown_connections 2024-05-08T22:31:49.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:31:49.159+0000 7f584a959700 1 -- 172.21.15.79:0/510238936 wait complete. 2024-05-08T22:31:49.384 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:49 smithi079 ceph-mon[46738]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:49.384 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.385 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.385 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:49.385 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:49.385 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.385 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.385 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:49 smithi062 ceph-mon[42571]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:49.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:49.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:49.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.552 DEBUG:teuthology.orchestra.run.smithi079:osd.2> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.2.service 2024-05-08T22:31:49.554 INFO:tasks.cephadm:Deploying osd.3 on smithi062 with /dev/vg_nvme/lv_3... 2024-05-08T22:31:49.554 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T22:31:49.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:49 smithi066 ceph-mon[46830]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:49.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:49.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:49.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:49.587 INFO:journalctl@ceph.osd.2.smithi079.stdout:-- Logs begin at Wed 2024-05-08 22:12:11 UTC. -- 2024-05-08T22:31:49.796 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:31:50.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:50 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:50.335 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:50 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:50.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:50 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:31:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T22:31:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T22:31:51.161 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-05-08T22:31:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-05-08T22:31:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011111 s, 944 MB/s 2024-05-08T22:31:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-05-08T22:31:51.317 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:31:50 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-2[52532]: 2024-05-08T22:31:50.890+0000 7fe80673d7c0 -1 osd.2 0 log_to_monitors true 2024-05-08T22:31:51.317 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:51 smithi079 ceph-mon[46738]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:51.317 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:51 smithi079 ceph-mon[46738]: from='osd.2 [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:31:51.317 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:51 smithi079 ceph-mon[46738]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:31:51.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:51 smithi062 ceph-mon[42571]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:51.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:51 smithi062 ceph-mon[42571]: from='osd.2 [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:31:51.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:51 smithi062 ceph-mon[42571]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:31:51.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:51 smithi066 ceph-mon[46830]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:51.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:51 smithi066 ceph-mon[46830]: from='osd.2 [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:31:51.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:51 smithi066 ceph-mon[46830]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T22:31:51.580 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2024-05-08T22:31:51.809 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:31:52.398 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: from='osd.2 [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi079 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi079 to 17114M 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:52.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:52.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:52.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: from='osd.2 [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi079 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi079 to 17114M 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:52.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:52.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T22:31:52.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T22:31:52.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: from='osd.2 [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:31:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:31:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi079 2024-05-08T22:31:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:31:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi079 to 17114M 2024-05-08T22:31:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:31:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.089+0000 7fbc03f9f700 1 -- 172.21.15.62:0/805262121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbfc1043c0 msgr2=0x7fbbfc1067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.089+0000 7fbc03f9f700 1 --2- 172.21.15.62:0/805262121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbfc1043c0 0x7fbbfc1067b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009990 tx=0x7fbbec009ca0 comp rx=0 tx=0).stop 2024-05-08T22:31:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.089+0000 7fbc03f9f700 1 -- 172.21.15.62:0/805262121 shutdown_connections 2024-05-08T22:31:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.089+0000 7fbc03f9f700 1 --2- 172.21.15.62:0/805262121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc106cf0 0x7fbbfc109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:53.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.089+0000 7fbc03f9f700 1 --2- 172.21.15.62:0/805262121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbfc101a90 0x7fbbfc103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.089+0000 7fbc03f9f700 1 --2- 172.21.15.62:0/805262121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbfc1043c0 0x7fbbfc1067b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.089+0000 7fbc03f9f700 1 -- 172.21.15.62:0/805262121 >> 172.21.15.62:0/805262121 conn(0x7fbbfc0fb4c0 msgr2=0x7fbbfc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.089+0000 7fbc03f9f700 1 -- 172.21.15.62:0/805262121 shutdown_connections 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.089+0000 7fbc03f9f700 1 -- 172.21.15.62:0/805262121 wait complete. 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.089+0000 7fbc03f9f700 1 Processor -- start 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc03f9f700 1 -- start start 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc03f9f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbfc101a90 0x7fbbfc1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc03f9f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbfc1043c0 0x7fbbfc0ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc03f9f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc106cf0 0x7fbbfc0fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc03f9f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc100720 con 0x7fbbfc1043c0 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc03f9f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc100890 con 0x7fbbfc106cf0 2024-05-08T22:31:53.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc03f9f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc1a0f20 con 0x7fbbfc101a90 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc01d3b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbfc101a90 0x7fbbfc1013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc0153a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbfc1043c0 0x7fbbfc0ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc01d3b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbfc101a90 0x7fbbfc1013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.62:45344/0 (socket says 172.21.15.62:45344) 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc01d3b700 1 -- 172.21.15.62:0/1886457424 learned_addr learned my addr 172.21.15.62:0/1886457424 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc0253c700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc106cf0 0x7fbbfc0fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc0153a700 1 -- 172.21.15.62:0/1886457424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbfc101a90 msgr2=0x7fbbfc1013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc0153a700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbfc101a90 0x7fbbfc1013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc0153a700 1 -- 172.21.15.62:0/1886457424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc106cf0 msgr2=0x7fbbfc0fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc0153a700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc106cf0 0x7fbbfc0fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc0153a700 1 -- 172.21.15.62:0/1886457424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec0095f0 con 0x7fbbfc1043c0 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc01d3b700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbfc101a90 0x7fbbfc1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:31:53.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.090+0000 7fbc0153a700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbfc1043c0 0x7fbbfc0ffa40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbbec000c00 tx=0x7fbbec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:31:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.091+0000 7fbbf2ffd700 1 -- 172.21.15.62:0/1886457424 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec01d070 con 0x7fbbfc1043c0 2024-05-08T22:31:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.091+0000 7fbbf2ffd700 1 -- 172.21.15.62:0/1886457424 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbec021410 con 0x7fbbfc1043c0 2024-05-08T22:31:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.091+0000 7fbc03f9f700 1 -- 172.21.15.62:0/1886457424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc1a1150 con 0x7fbbfc1043c0 2024-05-08T22:31:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.091+0000 7fbbf2ffd700 1 -- 172.21.15.62:0/1886457424 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec017650 con 0x7fbbfc1043c0 2024-05-08T22:31:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.091+0000 7fbc03f9f700 1 -- 172.21.15.62:0/1886457424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc1a15c0 con 0x7fbbfc1043c0 2024-05-08T22:31:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.092+0000 7fbc03f9f700 1 -- 172.21.15.62:0/1886457424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc0fd040 con 0x7fbbfc1043c0 2024-05-08T22:31:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.093+0000 7fbbf2ffd700 1 -- 172.21.15.62:0/1886457424 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 14) v1 ==== 135817+0+0 (secure 0 0 0) 0x7fbbec021580 con 0x7fbbfc1043c0 2024-05-08T22:31:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.093+0000 7fbbf2ffd700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbbe80a1110 0x7fbbe80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:31:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.093+0000 7fbbf2ffd700 1 -- 172.21.15.62:0/1886457424 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7fbbec0cc4f0 con 0x7fbbfc1043c0 2024-05-08T22:31:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.093+0000 7fbc01d3b700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbbe80a1110 0x7fbbe80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:31:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.094+0000 7fbc01d3b700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbbe80a1110 0x7fbbe80a35d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8011550 tx=0x7fbbf8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:31:53.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.098+0000 7fbbf2ffd700 1 -- 172.21.15.62:0/1886457424 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbbec097460 con 0x7fbbfc1043c0 2024-05-08T22:31:53.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:53.307+0000 7fbc03f9f700 1 -- 172.21.15.62:0/1886457424 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbfc0613e0 con 0x7fbbe80a1110 2024-05-08T22:31:53.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:53 smithi079 ceph-mon[46738]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:53.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:53 smithi079 ceph-mon[46738]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:31:53.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:53 smithi079 ceph-mon[46738]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T22:31:53.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:53.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:53.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:53 smithi079 ceph-mon[46738]: from='osd.2 ' entity='osd.2' 2024-05-08T22:31:53.389 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:31:52 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-2[52532]: 2024-05-08T22:31:52.949+0000 7fe7fb5a0700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T22:31:53.389 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:31:52 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-2[52532]: 2024-05-08T22:31:52.957+0000 7fe7f3b8e700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:31:53.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:53 smithi062 ceph-mon[42571]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:53.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:53 smithi062 ceph-mon[42571]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:31:53.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:53 smithi062 ceph-mon[42571]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T22:31:53.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:53.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:53.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:53 smithi062 ceph-mon[42571]: from='osd.2 ' entity='osd.2' 2024-05-08T22:31:53.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:53 smithi066 ceph-mon[46830]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:53.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:53 smithi066 ceph-mon[46830]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:31:53.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:53 smithi066 ceph-mon[46830]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T22:31:53.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:53.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:53.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:53 smithi066 ceph-mon[46830]: from='osd.2 ' entity='osd.2' 2024-05-08T22:31:54.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:54 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:31:54.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:54 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:31:54.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:54.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:54 smithi062 ceph-mon[42571]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:54.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:54.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:54.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:54.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:54 smithi062 ceph-mon[42571]: osd.2 [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835] boot 2024-05-08T22:31:54.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:54 smithi062 ceph-mon[42571]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T22:31:54.673 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:54.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:54 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:31:54.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:54 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:31:54.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:54.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:54 smithi066 ceph-mon[46830]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:54.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:54.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:54.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:54.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:54 smithi066 ceph-mon[46830]: osd.2 [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835] boot 2024-05-08T22:31:54.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:54 smithi066 ceph-mon[46830]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T22:31:54.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:54.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:54 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:31:54.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:54 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:31:54.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:54.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:54 smithi079 ceph-mon[46738]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:31:54.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:31:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:31:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:31:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:54 smithi079 ceph-mon[46738]: osd.2 [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835] boot 2024-05-08T22:31:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:54 smithi079 ceph-mon[46738]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T22:31:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T22:31:55.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:55 smithi062 ceph-mon[42571]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:55.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:31:55.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T22:31:55.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:55 smithi062 ceph-mon[42571]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T22:31:55.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:31:55.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:55 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/959067914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cb8e9ce-ae6a-4c93-a888-2e0c831691d9"}]: dispatch 2024-05-08T22:31:55.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:31:55.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:55 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/959067914' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cb8e9ce-ae6a-4c93-a888-2e0c831691d9"}]': finished 2024-05-08T22:31:55.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:55 smithi062 ceph-mon[42571]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T22:31:55.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:31:55.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:55 smithi066 ceph-mon[46830]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:55.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:31:55.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T22:31:55.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:55 smithi066 ceph-mon[46830]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T22:31:55.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:31:55.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:55 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/959067914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cb8e9ce-ae6a-4c93-a888-2e0c831691d9"}]: dispatch 2024-05-08T22:31:55.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:31:55.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:55 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/959067914' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cb8e9ce-ae6a-4c93-a888-2e0c831691d9"}]': finished 2024-05-08T22:31:55.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:55 smithi066 ceph-mon[46830]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T22:31:55.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:31:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:55 smithi079 ceph-mon[46738]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T22:31:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:31:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T22:31:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:55 smithi079 ceph-mon[46738]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T22:31:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:31:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:55 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/959067914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cb8e9ce-ae6a-4c93-a888-2e0c831691d9"}]: dispatch 2024-05-08T22:31:55.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:31:55.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:55 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/959067914' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cb8e9ce-ae6a-4c93-a888-2e0c831691d9"}]': finished 2024-05-08T22:31:55.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:55 smithi079 ceph-mon[46738]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T22:31:55.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:31:56.171 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:31:55 smithi062 sudo[68209]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:31:56.171 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:31:55 smithi062 sudo[68209]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:31:56.172 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:31:55 smithi062 sudo[68209]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:31:56.172 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:31:55 smithi062 sudo[68209]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:31:56.475 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:31:56 smithi062 sudo[68237]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:31:56.475 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:31:56 smithi062 sudo[68237]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:31:56.475 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:31:56 smithi062 sudo[68237]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:31:56.475 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:31:56 smithi062 sudo[68237]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:31:56.736 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:56 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2159310973' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:56.736 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:56 smithi066 ceph-mon[46830]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T22:31:56.736 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:31:56.736 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:56.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:56 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2159310973' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:56.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:56 smithi079 ceph-mon[46738]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T22:31:56.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:31:56.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:56.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:56 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2159310973' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:31:56.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:56 smithi062 ceph-mon[42571]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T22:31:56.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:31:56.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:57.073 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:31:56 smithi066 sudo[56629]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:31:57.073 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:31:56 smithi066 sudo[56629]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:31:57.073 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:31:56 smithi066 sudo[56629]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:31:57.073 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:31:56 smithi066 sudo[56629]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:31:57.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:31:57.483+0000 7fbbf2ffd700 1 -- 172.21.15.62:0/1886457424 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbbec097830 con 0x7fbbfc1043c0 2024-05-08T22:31:57.486 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:31:57 smithi066 sudo[56632]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:31:57.486 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:31:57 smithi066 sudo[56632]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:31:57.486 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:31:57 smithi066 sudo[56632]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:31:57.487 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:31:57 smithi066 sudo[56632]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:31:57.712 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:57 smithi079 ceph-mon[46738]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:31:57.712 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:57.713 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:31:57 smithi079 sudo[57989]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:31:57.713 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:31:57 smithi079 sudo[57989]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:31:57.713 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:31:57 smithi079 sudo[57989]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:31:57.713 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:31:57 smithi079 sudo[57989]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:31:57.713 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:31:57 smithi079 sudo[57992]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:31:57.713 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:31:57 smithi079 sudo[57992]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:31:57.713 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:31:57 smithi079 sudo[57992]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:31:57.713 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:31:57 smithi079 sudo[57992]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:31:57.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:57 smithi066 ceph-mon[46830]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:31:57.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:57.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:57 smithi062 ceph-mon[42571]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:31:57.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:58.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:58 smithi062 sudo[69015]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:31:58.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:58 smithi062 sudo[69015]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:31:58.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:58 smithi062 sudo[69015]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:31:58.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:58 smithi062 sudo[69015]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:31:58.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:58 smithi079 ceph-mon[46738]: mgrmap e15: x(active, since 113s), standbys: z, y 2024-05-08T22:31:58.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:58 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:58.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:58 smithi079 ceph-mon[46738]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:31:58.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:58 smithi079 sudo[57995]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:31:58.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:58 smithi079 sudo[57995]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:31:58.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:58 smithi079 sudo[57995]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:31:58.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:58 smithi079 sudo[57995]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:31:58.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:58 smithi062 sudo[69396]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:31:58.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:58 smithi062 sudo[69396]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:31:58.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:58 smithi062 sudo[69396]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:31:58.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:58 smithi062 sudo[69396]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:31:58.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:58 smithi062 ceph-mon[42571]: mgrmap e15: x(active, since 113s), standbys: z, y 2024-05-08T22:31:58.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:58 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:58.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:31:58 smithi062 ceph-mon[42571]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:31:59.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:58 smithi066 ceph-mon[46830]: mgrmap e15: x(active, since 113s), standbys: z, y 2024-05-08T22:31:59.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:58 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:31:59.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:58 smithi066 ceph-mon[46830]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:31:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:59 smithi079 sudo[57998]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:31:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:59 smithi079 sudo[57998]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:31:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:59 smithi079 sudo[57998]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:31:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:31:59 smithi079 sudo[57998]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:31:59.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:59 smithi066 sudo[56635]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T22:31:59.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:59 smithi066 sudo[56635]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:31:59.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:59 smithi066 sudo[56635]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:31:59.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:31:59 smithi066 sudo[56635]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:32:00.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:32:00.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:32:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:32:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:32:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:32:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:32:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:32:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:32:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:32:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:32:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:32:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:32:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:32:00.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 sudo[56638]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 sudo[56638]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 sudo[56638]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 sudo[56638]: pam_unix(sudo:session): session closed for user root 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:32:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:32:00.325 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:32:00.325 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:32:00.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:32:00.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:32:00.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:32:00.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:32:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:32:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:32:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:32:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:32:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:32:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:32:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T22:32:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T22:32:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T22:32:01.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:01 smithi066 ceph-mon[46830]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:32:01.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:01 smithi066 ceph-mon[46830]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:32:01.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:01 smithi066 ceph-mon[46830]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:32:01.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:01 smithi079 ceph-mon[46738]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:32:01.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:01 smithi079 ceph-mon[46738]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:32:01.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:01 smithi079 ceph-mon[46738]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:32:01.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:01 smithi062 ceph-mon[42571]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T22:32:01.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:01 smithi062 ceph-mon[42571]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T22:32:01.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:01 smithi062 ceph-mon[42571]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T22:32:02.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:02 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T22:32:02.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:02 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:02.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:02 smithi062 ceph-mon[42571]: Deploying daemon osd.3 on smithi062 2024-05-08T22:32:02.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:02 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T22:32:02.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:02 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:02.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:02 smithi066 ceph-mon[46830]: Deploying daemon osd.3 on smithi062 2024-05-08T22:32:02.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:02 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T22:32:02.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:02 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:02.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:02 smithi079 ceph-mon[46738]: Deploying daemon osd.3 on smithi062 2024-05-08T22:32:03.075 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:03 smithi062 ceph-mon[42571]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 326 KiB/s wr, 21 op/s 2024-05-08T22:32:03.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:03 smithi066 ceph-mon[46830]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 326 KiB/s wr, 21 op/s 2024-05-08T22:32:03.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:03 smithi079 ceph-mon[46738]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 326 KiB/s wr, 21 op/s 2024-05-08T22:32:05.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:05 smithi066 ceph-mon[46830]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-05-08T22:32:05.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:05.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:05.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:05.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:05 smithi079 ceph-mon[46738]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-05-08T22:32:05.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:05.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:05.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:05.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:05 smithi062 ceph-mon[42571]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-05-08T22:32:05.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:05.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:05.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:07.200 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 3 on host 'smithi062' 2024-05-08T22:32:07.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.199+0000 7fbbf2ffd700 1 -- 172.21.15.62:0/1886457424 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbbfc0613e0 con 0x7fbbe80a1110 2024-05-08T22:32:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 -- 172.21.15.62:0/1886457424 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbbe80a1110 msgr2=0x7fbbe80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbbe80a1110 0x7fbbe80a35d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8011550 tx=0x7fbbf8009260 comp rx=0 tx=0).stop 2024-05-08T22:32:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 -- 172.21.15.62:0/1886457424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbfc1043c0 msgr2=0x7fbbfc0ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:07.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbfc1043c0 0x7fbbfc0ffa40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbbec000c00 tx=0x7fbbec004730 comp rx=0 tx=0).stop 2024-05-08T22:32:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 -- 172.21.15.62:0/1886457424 shutdown_connections 2024-05-08T22:32:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbbe80a1110 0x7fbbe80a35d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc106cf0 0x7fbbfc0fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbfc101a90 0x7fbbfc1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 --2- 172.21.15.62:0/1886457424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbfc1043c0 0x7fbbfc0ffa40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 -- 172.21.15.62:0/1886457424 >> 172.21.15.62:0/1886457424 conn(0x7fbbfc0fb4c0 msgr2=0x7fbbfc103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 -- 172.21.15.62:0/1886457424 shutdown_connections 2024-05-08T22:32:07.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:07.202+0000 7fbc03f9f700 1 -- 172.21.15.62:0/1886457424 wait complete. 2024-05-08T22:32:07.742 DEBUG:teuthology.orchestra.run.smithi062:osd.3> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.3.service 2024-05-08T22:32:07.744 INFO:tasks.cephadm:Deploying osd.4 on smithi066 with /dev/vg_nvme/lv_3... 2024-05-08T22:32:07.744 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T22:32:07.775 INFO:journalctl@ceph.osd.3.smithi062.stdout:-- Logs begin at Wed 2024-05-08 22:12:07 UTC. -- 2024-05-08T22:32:07.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:07 smithi062 ceph-mon[42571]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2024-05-08T22:32:07.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:07.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:07.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:07.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:07.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:07.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:07.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:07.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:07.986 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:32:08.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:07 smithi066 ceph-mon[46830]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2024-05-08T22:32:08.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:08.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:08.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:08.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:08.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:08.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:08.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:08.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:08.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:07 smithi079 ceph-mon[46738]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2024-05-08T22:32:08.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:08.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:08.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:08.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:08.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:08.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:08.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:08.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:09.279 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T22:32:09.279 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T22:32:09.279 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-05-08T22:32:09.279 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-05-08T22:32:09.279 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110629 s, 948 MB/s 2024-05-08T22:32:09.280 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-05-08T22:32:09.422 INFO:journalctl@ceph.osd.3.smithi062.stdout:May 08 22:32:09 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-3[70463]: 2024-05-08T22:32:09.026+0000 7f5ee870c7c0 -1 osd.3 0 log_to_monitors true 2024-05-08T22:32:09.613 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_3 2024-05-08T22:32:09.839 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:32:10.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:09 smithi066 ceph-mon[46830]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-08T22:32:10.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:09 smithi066 ceph-mon[46830]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T22:32:10.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:10.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:10.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:10.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:10.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:09 smithi079 ceph-mon[46738]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-08T22:32:10.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:09 smithi079 ceph-mon[46738]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T22:32:10.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:10.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:10.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:10.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:10.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:09 smithi062 ceph-mon[42571]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-08T22:32:10.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:09 smithi062 ceph-mon[42571]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T22:32:10.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:10.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:10.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:10.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.906+0000 7f2e4645d700 1 -- 172.21.15.66:0/339740280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e40100fe0 msgr2=0x7f2e40101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.906+0000 7f2e4645d700 1 --2- 172.21.15.66:0/339740280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e40100fe0 0x7f2e40101400 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2e300099c0 tx=0x7f2e30009cd0 comp rx=0 tx=0).stop 2024-05-08T22:32:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.906+0000 7f2e4645d700 1 -- 172.21.15.66:0/339740280 shutdown_connections 2024-05-08T22:32:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.906+0000 7f2e4645d700 1 --2- 172.21.15.66:0/339740280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40102b80 0x7f2e40109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.906+0000 7f2e4645d700 1 --2- 172.21.15.66:0/339740280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e40100fe0 0x7f2e40101400 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.906+0000 7f2e4645d700 1 --2- 172.21.15.66:0/339740280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e401021e0 0x7f2e40102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.906+0000 7f2e4645d700 1 -- 172.21.15.66:0/339740280 >> 172.21.15.66:0/339740280 conn(0x7f2e400fc580 msgr2=0x7f2e400fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.906+0000 7f2e4645d700 1 -- 172.21.15.66:0/339740280 shutdown_connections 2024-05-08T22:32:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.906+0000 7f2e4645d700 1 -- 172.21.15.66:0/339740280 wait complete. 2024-05-08T22:32:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.907+0000 7f2e4645d700 1 Processor -- start 2024-05-08T22:32:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.907+0000 7f2e4645d700 1 -- start start 2024-05-08T22:32:10.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.907+0000 7f2e4645d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e40100fe0 0x7f2e40074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:10.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.907+0000 7f2e4645d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e401021e0 0x7f2e40073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:10.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.907+0000 7f2e4645d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40102b80 0x7f2e400738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:10.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.907+0000 7f2e4645d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40074070 con 0x7f2e401021e0 2024-05-08T22:32:10.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.907+0000 7f2e4645d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400741e0 con 0x7f2e40100fe0 2024-05-08T22:32:10.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.907+0000 7f2e4645d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e401a0f80 con 0x7f2e40102b80 2024-05-08T22:32:10.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.907+0000 7f2e3ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e40100fe0 0x7f2e40074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.907+0000 7f2e3ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e40100fe0 0x7f2e40074cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48474/0 (socket says 172.21.15.66:48474) 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.907+0000 7f2e3ffff700 1 -- 172.21.15.66:0/2048708405 learned_addr learned my addr 172.21.15.66:0/2048708405 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e3f7fe700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e401021e0 0x7f2e40073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e449fa700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40102b80 0x7f2e400738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e3ffff700 1 -- 172.21.15.66:0/2048708405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40102b80 msgr2=0x7f2e400738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e3ffff700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40102b80 0x7f2e400738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e3ffff700 1 -- 172.21.15.66:0/2048708405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e401021e0 msgr2=0x7f2e40073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e3ffff700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e401021e0 0x7f2e40073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e3ffff700 1 -- 172.21.15.66:0/2048708405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e300095f0 con 0x7f2e40100fe0 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e3ffff700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e40100fe0 0x7f2e40074cf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e30006010 tx=0x7f2e30004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e3d7fa700 1 -- 172.21.15.66:0/2048708405 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3001d070 con 0x7f2e40100fe0 2024-05-08T22:32:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e4645d700 1 -- 172.21.15.66:0/2048708405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e401a11b0 con 0x7f2e40100fe0 2024-05-08T22:32:10.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e3d7fa700 1 -- 172.21.15.66:0/2048708405 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e30003f90 con 0x7f2e40100fe0 2024-05-08T22:32:10.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e3d7fa700 1 -- 172.21.15.66:0/2048708405 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000fd70 con 0x7f2e40100fe0 2024-05-08T22:32:10.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.908+0000 7f2e4645d700 1 -- 172.21.15.66:0/2048708405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e401a15c0 con 0x7f2e40100fe0 2024-05-08T22:32:10.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.909+0000 7f2e4645d700 1 -- 172.21.15.66:0/2048708405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e40108460 con 0x7f2e40100fe0 2024-05-08T22:32:10.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.910+0000 7f2e3d7fa700 1 -- 172.21.15.66:0/2048708405 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2e3000f460 con 0x7f2e40100fe0 2024-05-08T22:32:10.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.911+0000 7f2e3d7fa700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e280a11a0 0x7f2e280a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:10.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.911+0000 7f2e3d7fa700 1 -- 172.21.15.66:0/2048708405 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3389+0+0 (secure 0 0 0) 0x7f2e300cc8f0 con 0x7f2e40100fe0 2024-05-08T22:32:10.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.911+0000 7f2e3f7fe700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e280a11a0 0x7f2e280a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:10.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.911+0000 7f2e3f7fe700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e280a11a0 0x7f2e280a3660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2e34007c60 tx=0x7f2e340073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:10.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:10.915+0000 7f2e3d7fa700 1 -- 172.21.15.66:0/2048708405 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2e300973b0 con 0x7f2e40100fe0 2024-05-08T22:32:11.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:10 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi062 2024-05-08T22:32:11.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:10 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi062 to 8557M 2024-05-08T22:32:11.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:10 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:11.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:10 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:11.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:10 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:11.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:10 smithi066 ceph-mon[46830]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T22:32:11.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:10 smithi066 ceph-mon[46830]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T22:32:11.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:10 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:11.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:10 smithi066 ceph-mon[46830]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:32:11.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:10 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:11.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:11.128+0000 7f2e4645d700 1 -- 172.21.15.66:0/2048708405 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e400613e0 con 0x7f2e280a11a0 2024-05-08T22:32:11.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:10 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi062 2024-05-08T22:32:11.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:10 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi062 to 8557M 2024-05-08T22:32:11.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:10 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:11.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:10 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:11.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:10 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:11.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:10 smithi079 ceph-mon[46738]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T22:32:11.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:10 smithi079 ceph-mon[46738]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T22:32:11.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:10 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:11.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:10 smithi079 ceph-mon[46738]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:32:11.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:10 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:11.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:10 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi062 2024-05-08T22:32:11.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:10 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi062 to 8557M 2024-05-08T22:32:11.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:10 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:11.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:10 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:11.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:10 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:11.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:10 smithi062 ceph-mon[42571]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T22:32:11.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:10 smithi062 ceph-mon[42571]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T22:32:11.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:10 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:11.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:10 smithi062 ceph-mon[42571]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:32:11.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:10 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:12.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:11 smithi066 ceph-mon[46830]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-08T22:32:12.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:11 smithi066 ceph-mon[46830]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:32:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:11 smithi066 ceph-mon[46830]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T22:32:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:32:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:32:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:11 smithi066 ceph-mon[46830]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' 2024-05-08T22:32:12.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:11 smithi079 ceph-mon[46738]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-08T22:32:12.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:11 smithi079 ceph-mon[46738]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:32:12.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:11 smithi079 ceph-mon[46738]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T22:32:12.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:12.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:12.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:32:12.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:32:12.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:12.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:11 smithi079 ceph-mon[46738]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' 2024-05-08T22:32:12.171 INFO:journalctl@ceph.osd.3.smithi062.stdout:May 08 22:32:11 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-3[70463]: 2024-05-08T22:32:11.730+0000 7f5edd56f700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T22:32:12.171 INFO:journalctl@ceph.osd.3.smithi062.stdout:May 08 22:32:11 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-3[70463]: 2024-05-08T22:32:11.741+0000 7f5ed5b5d700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:32:12.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:11 smithi062 ceph-mon[42571]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-05-08T22:32:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:11 smithi062 ceph-mon[42571]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:32:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:11 smithi062 ceph-mon[42571]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T22:32:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:32:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:32:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:11 smithi062 ceph-mon[42571]: from='osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230]' entity='osd.3' 2024-05-08T22:32:13.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:12 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:32:13.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:12 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:32:13.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:12 smithi066 ceph-mon[46830]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:32:13.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:12 smithi066 ceph-mon[46830]: osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230] boot 2024-05-08T22:32:13.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:12 smithi066 ceph-mon[46830]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T22:32:13.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:12 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:13.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:12 smithi066 ceph-mon[46830]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T22:32:13.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:12 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:32:13.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:12 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:32:13.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:12 smithi079 ceph-mon[46738]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:32:13.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:12 smithi079 ceph-mon[46738]: osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230] boot 2024-05-08T22:32:13.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:12 smithi079 ceph-mon[46738]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T22:32:13.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:12 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:13.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:12 smithi079 ceph-mon[46738]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T22:32:13.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:12 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:32:13.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:12 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:32:13.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:12 smithi062 ceph-mon[42571]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:32:13.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:12 smithi062 ceph-mon[42571]: osd.3 [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230] boot 2024-05-08T22:32:13.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:12 smithi062 ceph-mon[42571]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T22:32:13.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:12 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T22:32:13.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:12 smithi062 ceph-mon[42571]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T22:32:14.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:13 smithi079 ceph-mon[46738]: pgmap v78: 1 pgs: 1 peering; 577 KiB data, 373 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:14.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:13 smithi079 ceph-mon[46738]: from='client.? 172.21.15.66:0/2272942182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3e6d05-7c0c-4098-8e91-e91fce3173cd"}]: dispatch 2024-05-08T22:32:14.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:13 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3e6d05-7c0c-4098-8e91-e91fce3173cd"}]: dispatch 2024-05-08T22:32:14.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:13 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3e6d05-7c0c-4098-8e91-e91fce3173cd"}]': finished 2024-05-08T22:32:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:13 smithi079 ceph-mon[46738]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T22:32:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:13 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:13 smithi079 ceph-mon[46738]: from='client.? 172.21.15.66:0/1037477074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:32:14.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:13 smithi062 ceph-mon[42571]: pgmap v78: 1 pgs: 1 peering; 577 KiB data, 373 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:14.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:13 smithi062 ceph-mon[42571]: from='client.? 172.21.15.66:0/2272942182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3e6d05-7c0c-4098-8e91-e91fce3173cd"}]: dispatch 2024-05-08T22:32:14.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:13 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3e6d05-7c0c-4098-8e91-e91fce3173cd"}]: dispatch 2024-05-08T22:32:14.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:13 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3e6d05-7c0c-4098-8e91-e91fce3173cd"}]': finished 2024-05-08T22:32:14.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:13 smithi062 ceph-mon[42571]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T22:32:14.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:13 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:14.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:13 smithi062 ceph-mon[42571]: from='client.? 172.21.15.66:0/1037477074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:32:14.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:13 smithi066 ceph-mon[46830]: pgmap v78: 1 pgs: 1 peering; 577 KiB data, 373 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:14.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:13 smithi066 ceph-mon[46830]: from='client.? 172.21.15.66:0/2272942182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3e6d05-7c0c-4098-8e91-e91fce3173cd"}]: dispatch 2024-05-08T22:32:14.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:13 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3e6d05-7c0c-4098-8e91-e91fce3173cd"}]: dispatch 2024-05-08T22:32:14.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:13 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3e6d05-7c0c-4098-8e91-e91fce3173cd"}]': finished 2024-05-08T22:32:14.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:13 smithi066 ceph-mon[46830]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T22:32:14.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:13 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:14.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:13 smithi066 ceph-mon[46830]: from='client.? 172.21.15.66:0/1037477074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:32:15.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:14 smithi066 ceph-mon[46830]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T22:32:15.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:15.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:14 smithi066 ceph-mon[46830]: pgmap v82: 1 pgs: 1 peering; 577 KiB data, 372 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:15.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:14 smithi079 ceph-mon[46738]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T22:32:15.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:15.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:14 smithi079 ceph-mon[46738]: pgmap v82: 1 pgs: 1 peering; 577 KiB data, 372 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:14 smithi062 ceph-mon[42571]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T22:32:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:14 smithi062 ceph-mon[42571]: pgmap v82: 1 pgs: 1 peering; 577 KiB data, 372 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:16.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:15 smithi066 ceph-mon[46830]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T22:32:16.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:15 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:16.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:15 smithi079 ceph-mon[46738]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T22:32:16.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:15 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:16.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:15 smithi062 ceph-mon[42571]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T22:32:16.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:15 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:17.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:16 smithi066 ceph-mon[46830]: pgmap v84: 1 pgs: 1 peering; 577 KiB data, 372 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:17.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:16 smithi079 ceph-mon[46738]: pgmap v84: 1 pgs: 1 peering; 577 KiB data, 372 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:17.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:16 smithi062 ceph-mon[42571]: pgmap v84: 1 pgs: 1 peering; 577 KiB data, 372 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:19.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:19 smithi066 ceph-mon[46830]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-08T22:32:19.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:19 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T22:32:19.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:19 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:19.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:19 smithi079 ceph-mon[46738]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-08T22:32:19.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:19 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T22:32:19.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:19 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:19.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:19 smithi062 ceph-mon[42571]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-08T22:32:19.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:19 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T22:32:19.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:19 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:20.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:20 smithi066 ceph-mon[46830]: Deploying daemon osd.4 on smithi066 2024-05-08T22:32:20.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:20 smithi079 ceph-mon[46738]: Deploying daemon osd.4 on smithi066 2024-05-08T22:32:20.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:20 smithi062 ceph-mon[42571]: Deploying daemon osd.4 on smithi066 2024-05-08T22:32:21.134 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:21 smithi066 ceph-mon[46830]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-08T22:32:21.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:21 smithi079 ceph-mon[46738]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-08T22:32:21.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:21 smithi062 ceph-mon[42571]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-08T22:32:23.780 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:23 smithi066 ceph-mon[46830]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T22:32:23.780 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:23 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:23.780 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:23 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:23.781 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:23 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:23.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:23 smithi079 ceph-mon[46738]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T22:32:23.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:23 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:23.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:23 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:23.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:23 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:23.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:23 smithi062 ceph-mon[42571]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T22:32:23.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:23 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:23.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:23 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:23.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:23 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:24.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.680+0000 7f2e3d7fa700 1 -- 172.21.15.66:0/2048708405 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2e400613e0 con 0x7f2e280a11a0 2024-05-08T22:32:24.680 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 4 on host 'smithi066' 2024-05-08T22:32:24.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 -- 172.21.15.66:0/2048708405 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e280a11a0 msgr2=0x7f2e280a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:24.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e280a11a0 0x7f2e280a3660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2e34007c60 tx=0x7f2e340073d0 comp rx=0 tx=0).stop 2024-05-08T22:32:24.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 -- 172.21.15.66:0/2048708405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e40100fe0 msgr2=0x7f2e40074cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:24.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e40100fe0 0x7f2e40074cf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e30006010 tx=0x7f2e30004730 comp rx=0 tx=0).stop 2024-05-08T22:32:24.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 -- 172.21.15.66:0/2048708405 shutdown_connections 2024-05-08T22:32:24.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40102b80 0x7f2e400738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:24.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2e280a11a0 0x7f2e280a3660 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:24.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e40100fe0 0x7f2e40074cf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:24.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 --2- 172.21.15.66:0/2048708405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e401021e0 0x7f2e40073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:24.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 -- 172.21.15.66:0/2048708405 >> 172.21.15.66:0/2048708405 conn(0x7f2e400fc580 msgr2=0x7f2e401055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:24.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 -- 172.21.15.66:0/2048708405 shutdown_connections 2024-05-08T22:32:24.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:32:24.682+0000 7f2e4645d700 1 -- 172.21.15.66:0/2048708405 wait complete. 2024-05-08T22:32:24.946 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:24 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:24.946 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:24 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:24.946 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:24 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:24.946 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:24 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:24.947 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:24 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:24.947 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:24 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:24.947 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:24 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:24.947 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:24 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:25.092 DEBUG:teuthology.orchestra.run.smithi066:osd.4> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.4.service 2024-05-08T22:32:25.094 INFO:tasks.cephadm:Deploying osd.5 on smithi079 with /dev/vg_nvme/lv_3... 2024-05-08T22:32:25.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T22:32:25.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:24 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:25.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:24 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:25.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:24 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:25.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:24 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:25.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:24 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:25.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:24 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:25.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:24 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:25.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:24 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:25.131 INFO:journalctl@ceph.osd.4.smithi066.stdout:-- Logs begin at Wed 2024-05-08 22:13:17 UTC. -- 2024-05-08T22:32:25.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:25.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:25.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:25.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:25.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:25.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:25.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:25.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:25.327 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:32:25.760 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:25 smithi066 ceph-mon[46830]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:32:26.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:25 smithi079 ceph-mon[46738]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:32:26.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:25 smithi062 ceph-mon[42571]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:32:26.659 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T22:32:26.659 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T22:32:26.659 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-05-08T22:32:26.660 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-05-08T22:32:26.660 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120881 s, 867 MB/s 2024-05-08T22:32:26.660 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-05-08T22:32:26.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:26 smithi066 ceph-mon[46830]: from='osd.4 [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:32:26.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:26 smithi066 ceph-mon[46830]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:32:26.821 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 08 22:32:26 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-4[59782]: 2024-05-08T22:32:26.535+0000 7f2ee796d7c0 -1 osd.4 0 log_to_monitors true 2024-05-08T22:32:26.995 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:26 smithi079 ceph-mon[46738]: from='osd.4 [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:32:26.996 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:26 smithi079 ceph-mon[46738]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:32:27.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_3 2024-05-08T22:32:27.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:26 smithi062 ceph-mon[42571]: from='osd.4 [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:32:27.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:26 smithi062 ceph-mon[42571]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T22:32:27.341 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:32:28.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-08T22:32:28.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T22:32:28.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T22:32:28.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:28.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='osd.4 [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:32:28.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:32:28.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:28.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:28.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:28.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:28.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-08T22:32:28.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T22:32:28.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T22:32:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='osd.4 [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:32:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:32:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-08T22:32:28.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='osd.4 [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:28.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.362+0000 7f65aa74e700 1 -- 172.21.15.79:0/1388766907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a40ffe80 msgr2=0x7f65a41002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:28.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.362+0000 7f65aa74e700 1 --2- 172.21.15.79:0/1388766907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a40ffe80 0x7f65a41002a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6598009990 tx=0x7f6598009ca0 comp rx=0 tx=0).stop 2024-05-08T22:32:28.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.362+0000 7f65aa74e700 1 -- 172.21.15.79:0/1388766907 shutdown_connections 2024-05-08T22:32:28.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.362+0000 7f65aa74e700 1 --2- 172.21.15.79:0/1388766907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a4101a30 0x7f65a4108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:28.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.362+0000 7f65aa74e700 1 --2- 172.21.15.79:0/1388766907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a40ffe80 0x7f65a41002a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:28.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.362+0000 7f65aa74e700 1 --2- 172.21.15.79:0/1388766907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a4101070 0x7f65a41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:28.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.362+0000 7f65aa74e700 1 -- 172.21.15.79:0/1388766907 >> 172.21.15.79:0/1388766907 conn(0x7f65a40fb470 msgr2=0x7f65a40fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:28.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.362+0000 7f65aa74e700 1 -- 172.21.15.79:0/1388766907 shutdown_connections 2024-05-08T22:32:28.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.362+0000 7f65aa74e700 1 -- 172.21.15.79:0/1388766907 wait complete. 2024-05-08T22:32:28.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.362+0000 7f65aa74e700 1 Processor -- start 2024-05-08T22:32:28.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65aa74e700 1 -- start start 2024-05-08T22:32:28.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65aa74e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a40ffe80 0x7f65a4196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:28.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65aa74e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a4101070 0x7f65a4197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65aa74e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a4101a30 0x7f65a419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65aa74e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a419ea10 con 0x7f65a40ffe80 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65aa74e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a419eb50 con 0x7f65a4101070 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65aa74e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a419ece0 con 0x7f65a4101a30 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a8ceb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a4101a30 0x7f65a419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a8ceb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a4101a30 0x7f65a419e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42380/0 (socket says 172.21.15.79:42380) 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a4101070 0x7f65a4197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a8ceb700 1 -- 172.21.15.79:0/21603908 learned_addr learned my addr 172.21.15.79:0/21603908 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a3fff700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a40ffe80 0x7f65a4196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a8ceb700 1 -- 172.21.15.79:0/21603908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a4101070 msgr2=0x7f65a4197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a8ceb700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a4101070 0x7f65a4197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:28.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a8ceb700 1 -- 172.21.15.79:0/21603908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a40ffe80 msgr2=0x7f65a4196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:28.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a8ceb700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a40ffe80 0x7f65a4196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:28.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a8ceb700 1 -- 172.21.15.79:0/21603908 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65980095f0 con 0x7f65a4101a30 2024-05-08T22:32:28.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a8ceb700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a4101a30 0x7f65a419e440 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f659c007f10 tx=0x7f659c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:28.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.363+0000 7f65a17fa700 1 -- 172.21.15.79:0/21603908 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00dd20 con 0x7f65a4101a30 2024-05-08T22:32:28.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.364+0000 7f65a17fa700 1 -- 172.21.15.79:0/21603908 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f659c00f040 con 0x7f65a4101a30 2024-05-08T22:32:28.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.364+0000 7f65aa74e700 1 -- 172.21.15.79:0/21603908 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a419efc0 con 0x7f65a4101a30 2024-05-08T22:32:28.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.364+0000 7f65a17fa700 1 -- 172.21.15.79:0/21603908 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0048a0 con 0x7f65a4101a30 2024-05-08T22:32:28.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.364+0000 7f65aa74e700 1 -- 172.21.15.79:0/21603908 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a419f430 con 0x7f65a4101a30 2024-05-08T22:32:28.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.365+0000 7f65aa74e700 1 -- 172.21.15.79:0/21603908 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a404ec60 con 0x7f65a4101a30 2024-05-08T22:32:28.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.366+0000 7f65a17fa700 1 -- 172.21.15.79:0/21603908 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f659c003a00 con 0x7f65a4101a30 2024-05-08T22:32:28.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.366+0000 7f65a17fa700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f658c0a11a0 0x7f658c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:28.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.366+0000 7f65a3fff700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f658c0a11a0 0x7f658c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:28.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.366+0000 7f65a17fa700 1 -- 172.21.15.79:0/21603908 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3821+0+0 (secure 0 0 0) 0x7f659c0caf50 con 0x7f65a4101a30 2024-05-08T22:32:28.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.367+0000 7f65a3fff700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f658c0a11a0 0x7f658c0a3660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f659800b2e0 tx=0x7f659800b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:28.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.371+0000 7f65a17fa700 1 -- 172.21.15.79:0/21603908 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f659c095870 con 0x7f65a4101a30 2024-05-08T22:32:28.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:28.572+0000 7f65aa74e700 1 -- 172.21.15.79:0/21603908 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f65a4069dc0 con 0x7f658c0a11a0 2024-05-08T22:32:29.073 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 08 22:32:28 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-4[59782]: 2024-05-08T22:32:28.586+0000 7f2edc7d0700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T22:32:29.074 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 08 22:32:28 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-4[59782]: 2024-05-08T22:32:28.599+0000 7f2ed4dbe700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:32:29.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:28 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi066 2024-05-08T22:32:29.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:28 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi066 to 8556M 2024-05-08T22:32:29.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:28 smithi066 ceph-mon[46830]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:32:29.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:28 smithi066 ceph-mon[46830]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T22:32:29.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:28 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:29.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:28 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:29.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:28 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:32:29.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:28 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:32:29.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:28 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:29.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:28 smithi066 ceph-mon[46830]: from='osd.4 ' entity='osd.4' 2024-05-08T22:32:29.095 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:28 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi066 2024-05-08T22:32:29.096 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:28 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi066 to 8556M 2024-05-08T22:32:29.096 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:28 smithi079 ceph-mon[46738]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:32:29.096 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:28 smithi079 ceph-mon[46738]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T22:32:29.096 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:28 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:29.096 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:28 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:29.096 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:28 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:32:29.096 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:28 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:32:29.096 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:28 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:29.096 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:28 smithi079 ceph-mon[46738]: from='osd.4 ' entity='osd.4' 2024-05-08T22:32:29.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:28 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi066 2024-05-08T22:32:29.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:28 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi066 to 8556M 2024-05-08T22:32:29.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:28 smithi062 ceph-mon[42571]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:32:29.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:28 smithi062 ceph-mon[42571]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T22:32:29.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:28 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:29.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:28 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:29.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:28 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:32:29.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:28 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:32:29.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:28 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:29.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:28 smithi062 ceph-mon[42571]: from='osd.4 ' entity='osd.4' 2024-05-08T22:32:30.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:29 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:32:30.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:29 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:32:30.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:29 smithi066 ceph-mon[46830]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:30.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:29 smithi066 ceph-mon[46830]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:32:30.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:30.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:29 smithi066 ceph-mon[46830]: osd.4 [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866] boot 2024-05-08T22:32:30.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:29 smithi066 ceph-mon[46830]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T22:32:30.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:30.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:29 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:32:30.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:29 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:32:30.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:29 smithi079 ceph-mon[46738]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:30.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:29 smithi079 ceph-mon[46738]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:32:30.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:30.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:29 smithi079 ceph-mon[46738]: osd.4 [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866] boot 2024-05-08T22:32:30.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:29 smithi079 ceph-mon[46738]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T22:32:30.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:30.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:29 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:32:30.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:29 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:32:30.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:29 smithi062 ceph-mon[42571]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T22:32:30.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:29 smithi062 ceph-mon[42571]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:32:30.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:30.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:29 smithi062 ceph-mon[42571]: osd.4 [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866] boot 2024-05-08T22:32:30.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:29 smithi062 ceph-mon[42571]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T22:32:30.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T22:32:31.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:30 smithi066 ceph-mon[46830]: from='client.? 172.21.15.79:0/4060204630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7f834ac-4a24-4111-a581-4276baaa08a8"}]: dispatch 2024-05-08T22:32:31.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:30 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7f834ac-4a24-4111-a581-4276baaa08a8"}]: dispatch 2024-05-08T22:32:31.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:30 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7f834ac-4a24-4111-a581-4276baaa08a8"}]': finished 2024-05-08T22:32:31.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:30 smithi066 ceph-mon[46830]: osdmap e32: 6 total, 5 up, 6 in 2024-05-08T22:32:31.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:30 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:31.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:30 smithi079 ceph-mon[46738]: from='client.? 172.21.15.79:0/4060204630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7f834ac-4a24-4111-a581-4276baaa08a8"}]: dispatch 2024-05-08T22:32:31.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:30 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7f834ac-4a24-4111-a581-4276baaa08a8"}]: dispatch 2024-05-08T22:32:31.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:30 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7f834ac-4a24-4111-a581-4276baaa08a8"}]': finished 2024-05-08T22:32:31.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:30 smithi079 ceph-mon[46738]: osdmap e32: 6 total, 5 up, 6 in 2024-05-08T22:32:31.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:30 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:31.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:30 smithi062 ceph-mon[42571]: from='client.? 172.21.15.79:0/4060204630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7f834ac-4a24-4111-a581-4276baaa08a8"}]: dispatch 2024-05-08T22:32:31.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:30 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7f834ac-4a24-4111-a581-4276baaa08a8"}]: dispatch 2024-05-08T22:32:31.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:30 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7f834ac-4a24-4111-a581-4276baaa08a8"}]': finished 2024-05-08T22:32:31.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:30 smithi062 ceph-mon[42571]: osdmap e32: 6 total, 5 up, 6 in 2024-05-08T22:32:31.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:30 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:32.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:31 smithi066 ceph-mon[46830]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:32.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:31 smithi066 ceph-mon[46830]: from='client.? 172.21.15.79:0/2731652108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:32:32.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:31 smithi066 ceph-mon[46830]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T22:32:32.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:31 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:32.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:31 smithi079 ceph-mon[46738]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:32.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:31 smithi079 ceph-mon[46738]: from='client.? 172.21.15.79:0/2731652108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:32:32.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:31 smithi079 ceph-mon[46738]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T22:32:32.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:31 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:32.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:31 smithi062 ceph-mon[42571]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:32.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:31 smithi062 ceph-mon[42571]: from='client.? 172.21.15.79:0/2731652108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:32:32.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:31 smithi062 ceph-mon[42571]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T22:32:32.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:31 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:34.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:33 smithi066 ceph-mon[46830]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:33 smithi079 ceph-mon[46738]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:34.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:33 smithi062 ceph-mon[42571]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:35.849 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:35 smithi079 ceph-mon[46738]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:36.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:35 smithi066 ceph-mon[46830]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:36.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:35 smithi062 ceph-mon[42571]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:36.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T22:32:36.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:37.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T22:32:37.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:37.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T22:32:37.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:38.047 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:37 smithi079 ceph-mon[46738]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:38.047 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:37 smithi079 ceph-mon[46738]: Deploying daemon osd.5 on smithi079 2024-05-08T22:32:38.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:37 smithi066 ceph-mon[46830]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:38.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:37 smithi066 ceph-mon[46830]: Deploying daemon osd.5 on smithi079 2024-05-08T22:32:38.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:37 smithi062 ceph-mon[42571]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:38.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:37 smithi062 ceph-mon[42571]: Deploying daemon osd.5 on smithi079 2024-05-08T22:32:39.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:39 smithi079 ceph-mon[46738]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:40.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:39 smithi066 ceph-mon[46830]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:40.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:39 smithi062 ceph-mon[42571]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:41.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:41 smithi066 ceph-mon[46830]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:41.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:41 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:41.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:41 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:41.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:41 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:41.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:41 smithi079 ceph-mon[46738]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:41.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:41 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:41.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:41 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:41.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:41 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:41.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:41 smithi062 ceph-mon[42571]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:41.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:41 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:41.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:41 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:41.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:41 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:42.626 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:42 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:42.626 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:42 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:42.626 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:42 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:42.626 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:42 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:42.626 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:42 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:42.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.731+0000 7f65a17fa700 1 -- 172.21.15.79:0/21603908 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f65a4069dc0 con 0x7f658c0a11a0 2024-05-08T22:32:42.731 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 5 on host 'smithi079' 2024-05-08T22:32:42.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.733+0000 7f65aa74e700 1 -- 172.21.15.79:0/21603908 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f658c0a11a0 msgr2=0x7f658c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:42.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.733+0000 7f65aa74e700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f658c0a11a0 0x7f658c0a3660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f659800b2e0 tx=0x7f659800b750 comp rx=0 tx=0).stop 2024-05-08T22:32:42.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.733+0000 7f65aa74e700 1 -- 172.21.15.79:0/21603908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a4101a30 msgr2=0x7f65a419e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:42.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.733+0000 7f65aa74e700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a4101a30 0x7f65a419e440 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f659c007f10 tx=0x7f659c00d3b0 comp rx=0 tx=0).stop 2024-05-08T22:32:42.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.733+0000 7f65aa74e700 1 -- 172.21.15.79:0/21603908 shutdown_connections 2024-05-08T22:32:42.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.733+0000 7f65aa74e700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65a4101a30 0x7f65a419e440 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:42.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.733+0000 7f65aa74e700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a40ffe80 0x7f65a4196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:42.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.733+0000 7f65aa74e700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f658c0a11a0 0x7f658c0a3660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:42.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.733+0000 7f65aa74e700 1 --2- 172.21.15.79:0/21603908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a4101070 0x7f65a4197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:42.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.733+0000 7f65aa74e700 1 -- 172.21.15.79:0/21603908 >> 172.21.15.79:0/21603908 conn(0x7f65a40fb470 msgr2=0x7f65a40696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:42.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.734+0000 7f65aa74e700 1 -- 172.21.15.79:0/21603908 shutdown_connections 2024-05-08T22:32:42.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:32:42.734+0000 7f65aa74e700 1 -- 172.21.15.79:0/21603908 wait complete. 2024-05-08T22:32:42.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:42 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:42.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:42 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:42.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:42 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:42.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:42 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:42.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:42 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:42.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:42 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:42.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:42 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:42.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:42 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:42.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:42 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:42.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:42 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:43.115 DEBUG:teuthology.orchestra.run.smithi079:osd.5> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.5.service 2024-05-08T22:32:43.117 INFO:tasks.cephadm:Deploying osd.6 on smithi062 with /dev/vg_nvme/lv_2... 2024-05-08T22:32:43.117 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T22:32:43.149 INFO:journalctl@ceph.osd.5.smithi079.stdout:-- Logs begin at Wed 2024-05-08 22:12:11 UTC. -- 2024-05-08T22:32:43.357 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:32:43.758 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:43 smithi062 ceph-mon[42571]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:43.758 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:43.759 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:43.759 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:43.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:43 smithi079 ceph-mon[46738]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:43.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:43.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:43.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:44.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:43 smithi066 ceph-mon[46830]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:44.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:44.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:44.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:44.639 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:32:44 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:32:44.208+0000 7fce8ce8d7c0 -1 osd.5 0 log_to_monitors true 2024-05-08T22:32:44.671 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T22:32:44.671 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T22:32:44.672 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-05-08T22:32:44.672 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-05-08T22:32:44.672 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113187 s, 926 MB/s 2024-05-08T22:32:44.672 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 2024-05-08T22:32:44.672 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-05-08T22:32:45.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:44 smithi062 ceph-mon[42571]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:32:45.055 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2024-05-08T22:32:45.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:44 smithi066 ceph-mon[46830]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:32:45.132 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:44 smithi079 ceph-mon[46738]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T22:32:45.281 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:32:46.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:46.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T22:32:46.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T22:32:46.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:46.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:32:46.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:46.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:46.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:46.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:46.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.075 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:46.075 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T22:32:46.075 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T22:32:46.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:46.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:32:46.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:46.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:46.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:46.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:46.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.138 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:32:45 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:32:45.729+0000 7fce81cf0700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T22:32:46.138 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:32:45 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:32:45.739+0000 7fce7e2e6700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:32:46.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:46.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T22:32:46.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T22:32:46.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:46.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:32:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:32:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:32:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:46.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.256+0000 7f2d97c37700 1 -- 172.21.15.62:0/2089125517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d90101010 msgr2=0x7f2d90101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:46.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.256+0000 7f2d97c37700 1 --2- 172.21.15.62:0/2089125517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d90101010 0x7f2d90101430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2d800099c0 tx=0x7f2d80009cd0 comp rx=0 tx=0).stop 2024-05-08T22:32:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.256+0000 7f2d97c37700 1 -- 172.21.15.62:0/2089125517 shutdown_connections 2024-05-08T22:32:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.256+0000 7f2d97c37700 1 --2- 172.21.15.62:0/2089125517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d90102bb0 0x7f2d90109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.256+0000 7f2d97c37700 1 --2- 172.21.15.62:0/2089125517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d90101010 0x7f2d90101430 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.256+0000 7f2d97c37700 1 --2- 172.21.15.62:0/2089125517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d90102210 0x7f2d90102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.256+0000 7f2d97c37700 1 -- 172.21.15.62:0/2089125517 >> 172.21.15.62:0/2089125517 conn(0x7f2d900fc5b0 msgr2=0x7f2d900fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:32:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.256+0000 7f2d97c37700 1 -- 172.21.15.62:0/2089125517 shutdown_connections 2024-05-08T22:32:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.256+0000 7f2d97c37700 1 -- 172.21.15.62:0/2089125517 wait complete. 2024-05-08T22:32:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.257+0000 7f2d97c37700 1 Processor -- start 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.257+0000 7f2d97c37700 1 -- start start 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.257+0000 7f2d97c37700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d90101010 0x7f2d90196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.257+0000 7f2d97c37700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d90102210 0x7f2d901974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.257+0000 7f2d97c37700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d90102bb0 0x7f2d9019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.257+0000 7f2d97c37700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9019eaf0 con 0x7f2d90102210 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.257+0000 7f2d97c37700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9019ec30 con 0x7f2d90101010 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.257+0000 7f2d97c37700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9019edc0 con 0x7f2d90102bb0 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.257+0000 7f2d951d2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d90102210 0x7f2d901974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.257+0000 7f2d951d2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d90102210 0x7f2d901974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59344/0 (socket says 172.21.15.62:59344) 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.257+0000 7f2d951d2700 1 -- 172.21.15.62:0/95714503 learned_addr learned my addr 172.21.15.62:0/95714503 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d961d4700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d90102bb0 0x7f2d9019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d959d3700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d90101010 0x7f2d90196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d951d2700 1 -- 172.21.15.62:0/95714503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d90102bb0 msgr2=0x7f2d9019e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d951d2700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d90102bb0 0x7f2d9019e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d951d2700 1 -- 172.21.15.62:0/95714503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d90101010 msgr2=0x7f2d90196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:32:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d951d2700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d90101010 0x7f2d90196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:32:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d951d2700 1 -- 172.21.15.62:0/95714503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d800095f0 con 0x7f2d90102210 2024-05-08T22:32:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d951d2700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d90102210 0x7f2d901974f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00c950 tx=0x7f2d8c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d86ffd700 1 -- 172.21.15.62:0/95714503 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c00cea0 con 0x7f2d90102210 2024-05-08T22:32:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d97c37700 1 -- 172.21.15.62:0/95714503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9019f0a0 con 0x7f2d90102210 2024-05-08T22:32:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d86ffd700 1 -- 172.21.15.62:0/95714503 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d8c004830 con 0x7f2d90102210 2024-05-08T22:32:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d86ffd700 1 -- 172.21.15.62:0/95714503 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c005460 con 0x7f2d90102210 2024-05-08T22:32:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.258+0000 7f2d97c37700 1 -- 172.21.15.62:0/95714503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9019f5f0 con 0x7f2d90102210 2024-05-08T22:32:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.259+0000 7f2d97c37700 1 -- 172.21.15.62:0/95714503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d90108490 con 0x7f2d90102210 2024-05-08T22:32:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.260+0000 7f2d86ffd700 1 -- 172.21.15.62:0/95714503 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2d8c007ac0 con 0x7f2d90102210 2024-05-08T22:32:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.261+0000 7f2d86ffd700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2d7c0a11a0 0x7f2d7c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:32:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.261+0000 7f2d959d3700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2d7c0a11a0 0x7f2d7c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:32:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.261+0000 7f2d86ffd700 1 -- 172.21.15.62:0/95714503 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4253+0+0 (secure 0 0 0) 0x7f2d8c0cb9e0 con 0x7f2d90102210 2024-05-08T22:32:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.261+0000 7f2d959d3700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2d7c0a11a0 0x7f2d7c0a3660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000b580 tx=0x7f2d80005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:32:46.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.265+0000 7f2d86ffd700 1 -- 172.21.15.62:0/95714503 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2d8c0960b0 con 0x7f2d90102210 2024-05-08T22:32:46.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:32:46.478+0000 7f2d97c37700 1 -- 172.21.15.62:0/95714503 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d900613e0 con 0x7f2d7c0a11a0 2024-05-08T22:32:47.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi079 2024-05-08T22:32:47.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi079 to 8557M 2024-05-08T22:32:47.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:32:47.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T22:32:47.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:47.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:47.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:32:47.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:32:47.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:47.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211] boot 2024-05-08T22:32:47.013 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T22:32:47.013 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:46 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:47.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi079 2024-05-08T22:32:47.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi079 to 8557M 2024-05-08T22:32:47.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:32:47.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T22:32:47.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:47.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:47.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:32:47.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:32:47.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:47.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211] boot 2024-05-08T22:32:47.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T22:32:47.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:46 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:47.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi079 2024-05-08T22:32:47.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi079 to 8557M 2024-05-08T22:32:47.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: from='osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:32:47.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T22:32:47.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:47.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:47.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:32:47.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:32:47.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:47.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: osd.5 [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211] boot 2024-05-08T22:32:47.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T22:32:47.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:46 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T22:32:47.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:47 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:32:47.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:47 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:32:47.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:47 smithi062 ceph-mon[42571]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:47.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:47 smithi062 ceph-mon[42571]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:32:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:47 smithi062 ceph-mon[42571]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T22:32:48.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:47 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:32:48.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:47 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:32:48.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:47 smithi066 ceph-mon[46830]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:48.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:47 smithi066 ceph-mon[46830]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:32:48.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:47 smithi066 ceph-mon[46830]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T22:32:48.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:47 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:32:48.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:47 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:32:48.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:47 smithi079 ceph-mon[46738]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-08T22:32:48.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:47 smithi079 ceph-mon[46738]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:32:48.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:47 smithi079 ceph-mon[46738]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T22:32:49.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:48 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2779034286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da290a10-42f6-4e73-a6e6-e716af055721"}]: dispatch 2024-05-08T22:32:49.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:48 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2779034286' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da290a10-42f6-4e73-a6e6-e716af055721"}]': finished 2024-05-08T22:32:49.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:48 smithi066 ceph-mon[46830]: osdmap e38: 7 total, 6 up, 7 in 2024-05-08T22:32:49.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:48 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:32:49.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:48 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2779034286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da290a10-42f6-4e73-a6e6-e716af055721"}]: dispatch 2024-05-08T22:32:49.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:48 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2779034286' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da290a10-42f6-4e73-a6e6-e716af055721"}]': finished 2024-05-08T22:32:49.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:48 smithi079 ceph-mon[46738]: osdmap e38: 7 total, 6 up, 7 in 2024-05-08T22:32:49.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:48 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:32:49.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:48 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2779034286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da290a10-42f6-4e73-a6e6-e716af055721"}]: dispatch 2024-05-08T22:32:49.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:48 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2779034286' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da290a10-42f6-4e73-a6e6-e716af055721"}]': finished 2024-05-08T22:32:49.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:48 smithi062 ceph-mon[42571]: osdmap e38: 7 total, 6 up, 7 in 2024-05-08T22:32:49.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:48 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:32:50.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:49 smithi066 ceph-mon[46830]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:32:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:49 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/595069182' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:32:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:49 smithi066 ceph-mon[46830]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T22:32:50.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:32:50.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:49 smithi079 ceph-mon[46738]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:32:50.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:49 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/595069182' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:32:50.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:49 smithi079 ceph-mon[46738]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T22:32:50.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:32:50.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:49 smithi062 ceph-mon[42571]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:32:50.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:49 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/595069182' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:32:50.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:49 smithi062 ceph-mon[42571]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T22:32:50.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:32:51.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:51 smithi079 ceph-mon[46738]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:32:51.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:51 smithi079 ceph-mon[46738]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T22:32:51.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:32:51.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:51 smithi062 ceph-mon[42571]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:32:51.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:51 smithi062 ceph-mon[42571]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T22:32:51.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:32:51.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:51 smithi066 ceph-mon[46830]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:32:51.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:51 smithi066 ceph-mon[46830]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T22:32:51.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:32:53.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:53 smithi062 ceph-mon[42571]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-08T22:32:53.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:53 smithi066 ceph-mon[46830]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-08T22:32:53.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:53 smithi079 ceph-mon[46738]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-08T22:32:55.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:55 smithi062 ceph-mon[42571]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:32:55.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T22:32:55.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:55.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:55 smithi062 ceph-mon[42571]: Deploying daemon osd.6 on smithi062 2024-05-08T22:32:55.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:55 smithi066 ceph-mon[46830]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:32:55.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T22:32:55.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:55.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:55 smithi066 ceph-mon[46830]: Deploying daemon osd.6 on smithi062 2024-05-08T22:32:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:55 smithi079 ceph-mon[46738]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:32:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T22:32:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:32:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:55 smithi079 ceph-mon[46738]: Deploying daemon osd.6 on smithi062 2024-05-08T22:32:57.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:57 smithi062 ceph-mon[42571]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T22:32:57.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:57 smithi066 ceph-mon[46830]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T22:32:57.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:57 smithi079 ceph-mon[46738]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T22:32:59.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:59 smithi062 ceph-mon[42571]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T22:32:59.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:59.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:59.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:32:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:59 smithi066 ceph-mon[46830]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T22:32:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:32:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:59 smithi079 ceph-mon[46738]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T22:32:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:32:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:32:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:32:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.121+0000 7f2d86ffd700 1 -- 172.21.15.62:0/95714503 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2d900613e0 con 0x7f2d7c0a11a0 2024-05-08T22:33:00.122 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 6 on host 'smithi062' 2024-05-08T22:33:00.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 -- 172.21.15.62:0/95714503 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2d7c0a11a0 msgr2=0x7f2d7c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2d7c0a11a0 0x7f2d7c0a3660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000b580 tx=0x7f2d80005fb0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 -- 172.21.15.62:0/95714503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d90102210 msgr2=0x7f2d901974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:00.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d90102210 0x7f2d901974f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00c950 tx=0x7f2d8c00cc60 comp rx=0 tx=0).stop 2024-05-08T22:33:00.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 -- 172.21.15.62:0/95714503 shutdown_connections 2024-05-08T22:33:00.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d90102bb0 0x7f2d9019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2d7c0a11a0 0x7f2d7c0a3660 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d90101010 0x7f2d90196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 --2- 172.21.15.62:0/95714503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d90102210 0x7f2d901974f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:00.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 -- 172.21.15.62:0/95714503 >> 172.21.15.62:0/95714503 conn(0x7f2d900fc5b0 msgr2=0x7f2d901055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:00.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 -- 172.21.15.62:0/95714503 shutdown_connections 2024-05-08T22:33:00.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:00.124+0000 7f2d97c37700 1 -- 172.21.15.62:0/95714503 wait complete. 2024-05-08T22:33:00.521 DEBUG:teuthology.orchestra.run.smithi062:osd.6> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.6.service 2024-05-08T22:33:00.527 INFO:tasks.cephadm:Deploying osd.7 on smithi066 with /dev/vg_nvme/lv_2... 2024-05-08T22:33:00.527 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T22:33:00.559 INFO:journalctl@ceph.osd.6.smithi062.stdout:-- Logs begin at Wed 2024-05-08 22:12:07 UTC. -- 2024-05-08T22:33:00.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:00.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:00.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:00 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:00.767 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:33:00.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:00.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:00.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:00 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:00.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:00.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:00.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:00.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:00 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:01.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:01 smithi062 ceph-mon[42571]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:33:01.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:01 smithi066 ceph-mon[46830]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:33:01.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:01 smithi079 ceph-mon[46738]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:33:02.051 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T22:33:02.051 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T22:33:02.051 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-05-08T22:33:02.051 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-05-08T22:33:02.051 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127723 s, 821 MB/s 2024-05-08T22:33:02.052 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-05-08T22:33:02.242 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:33:01 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:33:01.989+0000 7f7ea9c4a7c0 -1 osd.6 0 log_to_monitors true 2024-05-08T22:33:02.377 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_2 2024-05-08T22:33:02.501 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:02 smithi062 ceph-mon[42571]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:33:02.607 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:33:02.707 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:02 smithi066 ceph-mon[46830]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:33:02.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:02 smithi079 ceph-mon[46738]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T22:33:03.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 -- 172.21.15.66:0/1533285405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d24106cf0 msgr2=0x7f5d24109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:03.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 --2- 172.21.15.66:0/1533285405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d24106cf0 0x7f5d24109160 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00b0a0 tx=0x7f5d1c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:33:03.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 -- 172.21.15.66:0/1533285405 shutdown_connections 2024-05-08T22:33:03.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 --2- 172.21.15.66:0/1533285405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d24106cf0 0x7f5d24109160 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:03.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 --2- 172.21.15.66:0/1533285405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d24101a90 0x7f5d24103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:03.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 --2- 172.21.15.66:0/1533285405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d241043c0 0x7f5d241067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:03.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 -- 172.21.15.66:0/1533285405 >> 172.21.15.66:0/1533285405 conn(0x7f5d240fb4c0 msgr2=0x7f5d240fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:03.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 -- 172.21.15.66:0/1533285405 shutdown_connections 2024-05-08T22:33:03.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 -- 172.21.15.66:0/1533285405 wait complete. 2024-05-08T22:33:03.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 Processor -- start 2024-05-08T22:33:03.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 -- start start 2024-05-08T22:33:03.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d24101a90 0x7f5d24196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:03.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d241043c0 0x7f5d241974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:03.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d24106cf0 0x7f5d2419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:03.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2419ead0 con 0x7f5d241043c0 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2419ec10 con 0x7f5d24101a90 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.483+0000 7f5d2ba07700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2419eda0 con 0x7f5d24106cf0 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d297a3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d24101a90 0x7f5d24196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d297a3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d24101a90 0x7f5d24196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48380/0 (socket says 172.21.15.66:48380) 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d297a3700 1 -- 172.21.15.66:0/274635628 learned_addr learned my addr 172.21.15.66:0/274635628 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d28fa2700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d241043c0 0x7f5d241974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d29fa4700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d24106cf0 0x7f5d2419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d297a3700 1 -- 172.21.15.66:0/274635628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d24106cf0 msgr2=0x7f5d2419e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d297a3700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d24106cf0 0x7f5d2419e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d297a3700 1 -- 172.21.15.66:0/274635628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d241043c0 msgr2=0x7f5d241974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d297a3700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d241043c0 0x7f5d241974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d297a3700 1 -- 172.21.15.66:0/274635628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c009d00 con 0x7f5d24101a90 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d297a3700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d24101a90 0x7f5d24196f90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000d980 tx=0x7f5d2000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d1a7fc700 1 -- 172.21.15.66:0/274635628 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20004220 con 0x7f5d24101a90 2024-05-08T22:33:03.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d1a7fc700 1 -- 172.21.15.66:0/274635628 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d20007640 con 0x7f5d24101a90 2024-05-08T22:33:03.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d1a7fc700 1 -- 172.21.15.66:0/274635628 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d200056c0 con 0x7f5d24101a90 2024-05-08T22:33:03.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d2ba07700 1 -- 172.21.15.66:0/274635628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2419f080 con 0x7f5d24101a90 2024-05-08T22:33:03.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.484+0000 7f5d2ba07700 1 -- 172.21.15.66:0/274635628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2419f580 con 0x7f5d24101a90 2024-05-08T22:33:03.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.485+0000 7f5d2ba07700 1 -- 172.21.15.66:0/274635628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d240fd040 con 0x7f5d24101a90 2024-05-08T22:33:03.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.486+0000 7f5d1a7fc700 1 -- 172.21.15.66:0/274635628 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5d20004380 con 0x7f5d24101a90 2024-05-08T22:33:03.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.487+0000 7f5d1a7fc700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5d100a11a0 0x7f5d100a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:03.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.488+0000 7f5d1a7fc700 1 -- 172.21.15.66:0/274635628 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4685+0+0 (secure 0 0 0) 0x7f5d200cbd90 con 0x7f5d24101a90 2024-05-08T22:33:03.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.488+0000 7f5d28fa2700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5d100a11a0 0x7f5d100a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:03.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.488+0000 7f5d28fa2700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5d100a11a0 0x7f5d100a3660 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400abd0 tx=0x7f5d1400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:03.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.491+0000 7f5d1a7fc700 1 -- 172.21.15.66:0/274635628 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5d200962b0 con 0x7f5d24101a90 2024-05-08T22:33:03.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T22:33:03.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T22:33:03.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T22:33:03.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi062 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi062 to 5704M 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:03.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:03.702+0000 7f5d2ba07700 1 -- 172.21.15.66:0/274635628 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d2402ce10 con 0x7f5d100a11a0 2024-05-08T22:33:03.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T22:33:03.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T22:33:03.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T22:33:03.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi062 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi062 to 5704M 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:03.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:03.921 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:33:03 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:33:03.458+0000 7f7e9eaad700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T22:33:03.922 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:33:03 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:33:03.473+0000 7f7e9b0a3700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:03.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T22:33:03.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T22:33:03.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T22:33:03.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:03.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:33:03.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi062 2024-05-08T22:33:03.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:03.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:03.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:03.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:03.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:03.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi062 to 5704M 2024-05-08T22:33:03.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:03.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:03.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:03.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:04.725 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:04 smithi066 ceph-mon[46830]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:33:04.725 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:04 smithi066 ceph-mon[46830]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T22:33:04.726 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:04.726 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:04.726 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:04 smithi066 ceph-mon[46830]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:04.726 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:04.726 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:04.726 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:04.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:04 smithi079 ceph-mon[46738]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:33:04.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:04 smithi079 ceph-mon[46738]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T22:33:04.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:04.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:04.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:04 smithi079 ceph-mon[46738]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:04.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:04.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:04.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:04.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:04 smithi062 ceph-mon[42571]: from='osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:33:04.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:04 smithi062 ceph-mon[42571]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T22:33:04.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:04.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:04.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:04 smithi062 ceph-mon[42571]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:04.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:04.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:04.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:05.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:05 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:33:05.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:05 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:33:05.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:05 smithi066 ceph-mon[46830]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:33:05.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:05.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:05 smithi066 ceph-mon[46830]: osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145] boot 2024-05-08T22:33:05.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:05 smithi066 ceph-mon[46830]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T22:33:05.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:05.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:05 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:33:05.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:05 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:33:05.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:05 smithi079 ceph-mon[46738]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:33:05.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:05.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:05 smithi079 ceph-mon[46738]: osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145] boot 2024-05-08T22:33:05.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:05 smithi079 ceph-mon[46738]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T22:33:05.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:05.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:05 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:33:05.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:05 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:33:05.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:05 smithi062 ceph-mon[42571]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T22:33:05.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:05.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:05 smithi062 ceph-mon[42571]: osd.6 [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145] boot 2024-05-08T22:33:05.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:05 smithi062 ceph-mon[42571]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T22:33:05.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T22:33:06.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:06 smithi066 ceph-mon[46830]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T22:33:06.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:06 smithi066 ceph-mon[46830]: from='client.? 172.21.15.66:0/2993162461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee23b28b-603a-4f50-b3f1-2404bef4fa74"}]: dispatch 2024-05-08T22:33:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:06 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee23b28b-603a-4f50-b3f1-2404bef4fa74"}]: dispatch 2024-05-08T22:33:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:06 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee23b28b-603a-4f50-b3f1-2404bef4fa74"}]': finished 2024-05-08T22:33:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:06 smithi066 ceph-mon[46830]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T22:33:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:06 smithi066 ceph-mon[46830]: from='client.? 172.21.15.66:0/1827683349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:33:06.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:06 smithi079 ceph-mon[46738]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T22:33:06.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:06 smithi079 ceph-mon[46738]: from='client.? 172.21.15.66:0/2993162461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee23b28b-603a-4f50-b3f1-2404bef4fa74"}]: dispatch 2024-05-08T22:33:06.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:06 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee23b28b-603a-4f50-b3f1-2404bef4fa74"}]: dispatch 2024-05-08T22:33:06.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:06 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee23b28b-603a-4f50-b3f1-2404bef4fa74"}]': finished 2024-05-08T22:33:06.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:06 smithi079 ceph-mon[46738]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T22:33:06.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:06.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:06 smithi079 ceph-mon[46738]: from='client.? 172.21.15.66:0/1827683349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:33:06.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:06 smithi062 ceph-mon[42571]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T22:33:06.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:06 smithi062 ceph-mon[42571]: from='client.? 172.21.15.66:0/2993162461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee23b28b-603a-4f50-b3f1-2404bef4fa74"}]: dispatch 2024-05-08T22:33:06.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:06 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee23b28b-603a-4f50-b3f1-2404bef4fa74"}]: dispatch 2024-05-08T22:33:06.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:06 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee23b28b-603a-4f50-b3f1-2404bef4fa74"}]': finished 2024-05-08T22:33:06.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:06 smithi062 ceph-mon[42571]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T22:33:06.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:06.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:06 smithi062 ceph-mon[42571]: from='client.? 172.21.15.66:0/1827683349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:33:07.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:07 smithi066 ceph-mon[46830]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:07.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:07 smithi079 ceph-mon[46738]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:07.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:07 smithi062 ceph-mon[42571]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:09.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:09 smithi066 ceph-mon[46830]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:09 smithi079 ceph-mon[46738]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:09.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:09 smithi062 ceph-mon[42571]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:11 smithi066 ceph-mon[46830]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:11.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:11 smithi079 ceph-mon[46738]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:11.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:11 smithi062 ceph-mon[42571]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:12.716 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:12 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T22:33:12.716 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:12 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:12.716 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:12 smithi066 ceph-mon[46830]: Deploying daemon osd.7 on smithi066 2024-05-08T22:33:12.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:12 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T22:33:12.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:12 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:12.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:12 smithi079 ceph-mon[46738]: Deploying daemon osd.7 on smithi066 2024-05-08T22:33:12.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:12 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T22:33:12.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:12 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:12.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:12 smithi062 ceph-mon[42571]: Deploying daemon osd.7 on smithi066 2024-05-08T22:33:13.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:13 smithi066 ceph-mon[46830]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:13.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:13 smithi079 ceph-mon[46738]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:13.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:13 smithi062 ceph-mon[42571]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:15.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:15 smithi079 ceph-mon[46738]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:15.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:15 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:15.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:15 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:15.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:15 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:15.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:15 smithi062 ceph-mon[42571]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:15.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:15 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:15.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:15 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:15.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:15 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:15.954 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:15 smithi066 ceph-mon[46830]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:15.954 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:15 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:15.954 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:15 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:15.954 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:15 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:17.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.555+0000 7f5d1a7fc700 1 -- 172.21.15.66:0/274635628 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5d2402ce10 con 0x7f5d100a11a0 2024-05-08T22:33:17.556 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 7 on host 'smithi066' 2024-05-08T22:33:17.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 -- 172.21.15.66:0/274635628 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5d100a11a0 msgr2=0x7f5d100a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:17.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5d100a11a0 0x7f5d100a3660 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400abd0 tx=0x7f5d1400a380 comp rx=0 tx=0).stop 2024-05-08T22:33:17.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 -- 172.21.15.66:0/274635628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d24101a90 msgr2=0x7f5d24196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:17.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d24101a90 0x7f5d24196f90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000d980 tx=0x7f5d2000dc90 comp rx=0 tx=0).stop 2024-05-08T22:33:17.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 -- 172.21.15.66:0/274635628 shutdown_connections 2024-05-08T22:33:17.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d24106cf0 0x7f5d2419e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:17.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5d100a11a0 0x7f5d100a3660 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:17.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d24101a90 0x7f5d24196f90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:17.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 --2- 172.21.15.66:0/274635628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d241043c0 0x7f5d241974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:17.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 -- 172.21.15.66:0/274635628 >> 172.21.15.66:0/274635628 conn(0x7f5d240fb4c0 msgr2=0x7f5d24102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:17.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 -- 172.21.15.66:0/274635628 shutdown_connections 2024-05-08T22:33:17.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:17.557+0000 7f5d2ba07700 1 -- 172.21.15.66:0/274635628 wait complete. 2024-05-08T22:33:17.839 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:17 smithi066 ceph-mon[46830]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:17.839 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.839 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.839 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:17.839 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:17.839 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.839 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.840 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.840 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:17.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:17 smithi079 ceph-mon[46738]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:17.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:17.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:17.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:17.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:17 smithi062 ceph-mon[42571]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:17.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:17.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:17.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:17.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:17.944 DEBUG:teuthology.orchestra.run.smithi066:osd.7> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.7.service 2024-05-08T22:33:17.946 INFO:tasks.cephadm:Deploying osd.8 on smithi079 with /dev/vg_nvme/lv_2... 2024-05-08T22:33:17.946 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T22:33:17.976 INFO:journalctl@ceph.osd.7.smithi066.stdout:-- Logs begin at Wed 2024-05-08 22:13:17 UTC. -- 2024-05-08T22:33:18.178 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:33:19.522 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T22:33:19.522 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T22:33:19.522 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-05-08T22:33:19.522 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-05-08T22:33:19.522 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112809 s, 930 MB/s 2024-05-08T22:33:19.522 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-05-08T22:33:19.573 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:33:19 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:33:19.208+0000 7fa1dd5787c0 -1 osd.7 0 log_to_monitors true 2024-05-08T22:33:19.867 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:19 smithi066 ceph-mon[46830]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:19.867 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:19 smithi066 ceph-mon[46830]: from='osd.7 [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:33:19.867 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:19 smithi066 ceph-mon[46830]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:33:19.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:19 smithi079 ceph-mon[46738]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:19.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:19 smithi079 ceph-mon[46738]: from='osd.7 [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:33:19.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:19 smithi079 ceph-mon[46738]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:33:19.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:19 smithi062 ceph-mon[42571]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:19.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:19 smithi062 ceph-mon[42571]: from='osd.7 [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:33:19.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:19 smithi062 ceph-mon[42571]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T22:33:19.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_2 2024-05-08T22:33:20.157 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:33:20.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T22:33:20.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T22:33:20.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:20.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='osd.7 [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:33:20.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:33:20.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi066 2024-05-08T22:33:20.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:20.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:20.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:20.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:20.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:20.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:20.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:20.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:20.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:20 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:20.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T22:33:20.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T22:33:20.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:20.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='osd.7 [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:33:20.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:33:20.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi066 2024-05-08T22:33:20.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:20.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:20.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:20.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:20.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:20.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:20.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:20.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:20.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:21.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T22:33:21.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T22:33:21.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='osd.7 [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi066 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:21.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:20 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:21.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.200+0000 7f7c444ef700 1 -- 172.21.15.79:0/1611646583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c3c101060 msgr2=0x7f7c3c1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.200+0000 7f7c444ef700 1 --2- 172.21.15.79:0/1611646583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c3c101060 0x7f7c3c1014e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009990 tx=0x7f7c38009ca0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.201+0000 7f7c444ef700 1 -- 172.21.15.79:0/1611646583 shutdown_connections 2024-05-08T22:33:21.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.201+0000 7f7c444ef700 1 --2- 172.21.15.79:0/1611646583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c101a20 0x7f7c3c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.201+0000 7f7c444ef700 1 --2- 172.21.15.79:0/1611646583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c3c0fff00 0x7f7c3c100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.201+0000 7f7c444ef700 1 --2- 172.21.15.79:0/1611646583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c3c101060 0x7f7c3c1014e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.201+0000 7f7c444ef700 1 -- 172.21.15.79:0/1611646583 >> 172.21.15.79:0/1611646583 conn(0x7f7c3c0fb4c0 msgr2=0x7f7c3c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:21.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.201+0000 7f7c444ef700 1 -- 172.21.15.79:0/1611646583 shutdown_connections 2024-05-08T22:33:21.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.201+0000 7f7c444ef700 1 -- 172.21.15.79:0/1611646583 wait complete. 2024-05-08T22:33:21.203 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.201+0000 7f7c444ef700 1 Processor -- start 2024-05-08T22:33:21.203 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.201+0000 7f7c444ef700 1 -- start start 2024-05-08T22:33:21.203 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c444ef700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c3c0fff00 0x7f7c3c069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.203 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c444ef700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c101060 0x7f7c3c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.203 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c444ef700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c3c101a20 0x7f7c3c19e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c444ef700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c19e970 con 0x7f7c3c101060 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c444ef700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c19eae0 con 0x7f7c3c101a20 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c444ef700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c19eca0 con 0x7f7c3c0fff00 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c4228b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c3c0fff00 0x7f7c3c069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c4228b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c3c0fff00 0x7f7c3c069d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:59486/0 (socket says 172.21.15.79:59486) 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c4228b700 1 -- 172.21.15.79:0/820338576 learned_addr learned my addr 172.21.15.79:0/820338576 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c42a8c700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c3c101a20 0x7f7c3c19e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c41a8a700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c101060 0x7f7c3c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c4228b700 1 -- 172.21.15.79:0/820338576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c3c101a20 msgr2=0x7f7c3c19e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c4228b700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c3c101a20 0x7f7c3c19e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c4228b700 1 -- 172.21.15.79:0/820338576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c101060 msgr2=0x7f7c3c19bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:21.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c4228b700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c101060 0x7f7c3c19bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:21.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c4228b700 1 -- 172.21.15.79:0/820338576 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380095f0 con 0x7f7c3c0fff00 2024-05-08T22:33:21.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c4228b700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c3c0fff00 0x7f7c3c069d10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000e9a0 tx=0x7f7c3000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c2f7fe700 1 -- 172.21.15.79:0/820338576 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30009790 con 0x7f7c3c0fff00 2024-05-08T22:33:21.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c2f7fe700 1 -- 172.21.15.79:0/820338576 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c3000cbd0 con 0x7f7c3c0fff00 2024-05-08T22:33:21.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c444ef700 1 -- 172.21.15.79:0/820338576 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3c19ef80 con 0x7f7c3c0fff00 2024-05-08T22:33:21.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c2f7fe700 1 -- 172.21.15.79:0/820338576 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30018680 con 0x7f7c3c0fff00 2024-05-08T22:33:21.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.202+0000 7f7c444ef700 1 -- 172.21.15.79:0/820338576 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c19f390 con 0x7f7c3c0fff00 2024-05-08T22:33:21.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.203+0000 7f7c444ef700 1 -- 172.21.15.79:0/820338576 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3c1073d0 con 0x7f7c3c0fff00 2024-05-08T22:33:21.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.204+0000 7f7c2f7fe700 1 -- 172.21.15.79:0/820338576 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7c3000cd40 con 0x7f7c3c0fff00 2024-05-08T22:33:21.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.205+0000 7f7c2f7fe700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7c280a1120 0x7f7c280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:21.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.205+0000 7f7c2f7fe700 1 -- 172.21.15.79:0/820338576 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f7c30014070 con 0x7f7c3c0fff00 2024-05-08T22:33:21.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.205+0000 7f7c41a8a700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7c280a1120 0x7f7c280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:21.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.206+0000 7f7c41a8a700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7c280a1120 0x7f7c280a35e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800b2e0 tx=0x7f7c38005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:21.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.209+0000 7f7c2f7fe700 1 -- 172.21.15.79:0/820338576 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7c30052d60 con 0x7f7c3c0fff00 2024-05-08T22:33:21.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:21.411+0000 7f7c444ef700 1 -- 172.21.15.79:0/820338576 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c3c02cee0 con 0x7f7c280a1120 2024-05-08T22:33:21.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:33:21 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:33:21.458+0000 7fa1d23db700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T22:33:21.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:33:21 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:33:21.474+0000 7fa1ca9c9700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi066 to 5704M 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: from='osd.7 ' entity='osd.7' 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: osd.7 [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757] boot 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T22:33:21.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:21 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:21.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi066 to 5704M 2024-05-08T22:33:21.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:21.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:33:21.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T22:33:21.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:21.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:21.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:21.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:21.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:21.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: from='osd.7 ' entity='osd.7' 2024-05-08T22:33:21.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: osd.7 [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757] boot 2024-05-08T22:33:21.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T22:33:21.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:21 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:21.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi066 to 5704M 2024-05-08T22:33:21.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T22:33:21.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:33:21.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T22:33:21.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:21.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:21.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:21.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:21.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:21.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: from='osd.7 ' entity='osd.7' 2024-05-08T22:33:21.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: osd.7 [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757] boot 2024-05-08T22:33:21.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T22:33:21.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:21 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T22:33:22.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:22 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:33:22.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:22 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:33:22.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:22 smithi079 ceph-mon[46738]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:22.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:22 smithi079 ceph-mon[46738]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T22:33:22.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:22 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:33:22.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:22 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:33:22.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:22 smithi062 ceph-mon[42571]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:22.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:22 smithi062 ceph-mon[42571]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T22:33:23.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:22 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:33:23.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:22 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:33:23.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:22 smithi066 ceph-mon[46830]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:23.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:22 smithi066 ceph-mon[46830]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T22:33:23.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:23 smithi079 ceph-mon[46738]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:23.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:23 smithi079 ceph-mon[46738]: from='client.? 172.21.15.79:0/3601529064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f193ce6-5093-45c8-aee0-2c6e99873111"}]: dispatch 2024-05-08T22:33:23.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:23 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f193ce6-5093-45c8-aee0-2c6e99873111"}]: dispatch 2024-05-08T22:33:23.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:23 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f193ce6-5093-45c8-aee0-2c6e99873111"}]': finished 2024-05-08T22:33:23.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:23 smithi079 ceph-mon[46738]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T22:33:23.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:23 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:23.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:23 smithi062 ceph-mon[42571]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:23.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:23 smithi062 ceph-mon[42571]: from='client.? 172.21.15.79:0/3601529064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f193ce6-5093-45c8-aee0-2c6e99873111"}]: dispatch 2024-05-08T22:33:23.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:23 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f193ce6-5093-45c8-aee0-2c6e99873111"}]: dispatch 2024-05-08T22:33:23.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:23 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f193ce6-5093-45c8-aee0-2c6e99873111"}]': finished 2024-05-08T22:33:23.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:23 smithi062 ceph-mon[42571]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T22:33:23.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:23 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:24.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:23 smithi066 ceph-mon[46830]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:24.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:23 smithi066 ceph-mon[46830]: from='client.? 172.21.15.79:0/3601529064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f193ce6-5093-45c8-aee0-2c6e99873111"}]: dispatch 2024-05-08T22:33:24.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:23 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f193ce6-5093-45c8-aee0-2c6e99873111"}]: dispatch 2024-05-08T22:33:24.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:23 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f193ce6-5093-45c8-aee0-2c6e99873111"}]': finished 2024-05-08T22:33:24.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:23 smithi066 ceph-mon[46830]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T22:33:24.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:23 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:24.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:24 smithi079 ceph-mon[46738]: from='client.? 172.21.15.79:0/1707260028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:33:24.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:24 smithi079 ceph-mon[46738]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T22:33:24.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:24 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:24.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:24 smithi062 ceph-mon[42571]: from='client.? 172.21.15.79:0/1707260028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:33:24.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:24 smithi062 ceph-mon[42571]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T22:33:24.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:25.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:24 smithi066 ceph-mon[46830]: from='client.? 172.21.15.79:0/1707260028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:33:25.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:24 smithi066 ceph-mon[46830]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T22:33:25.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:24 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:25.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:25 smithi079 ceph-mon[46738]: pgmap v140: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:25.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:25 smithi079 ceph-mon[46738]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T22:33:25.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:25 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:25.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:25 smithi062 ceph-mon[42571]: pgmap v140: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:25.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:25 smithi062 ceph-mon[42571]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T22:33:25.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:25 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:26.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:25 smithi066 ceph-mon[46830]: pgmap v140: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:26.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:25 smithi066 ceph-mon[46830]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T22:33:26.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:25 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:27.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:27 smithi079 ceph-mon[46738]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:27.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:27 smithi062 ceph-mon[42571]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:28.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:27 smithi066 ceph-mon[46830]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:29.878 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:29 smithi079 ceph-mon[46738]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:29.878 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T22:33:29.878 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:29.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:29 smithi062 ceph-mon[42571]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:29.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T22:33:29.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:30.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:29 smithi066 ceph-mon[46830]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:30.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T22:33:30.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:30.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:30 smithi079 ceph-mon[46738]: Deploying daemon osd.8 on smithi079 2024-05-08T22:33:30.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:30 smithi062 ceph-mon[42571]: Deploying daemon osd.8 on smithi079 2024-05-08T22:33:31.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:30 smithi066 ceph-mon[46830]: Deploying daemon osd.8 on smithi079 2024-05-08T22:33:31.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:31 smithi079 ceph-mon[46738]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:31.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:31 smithi062 ceph-mon[42571]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:32.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:31 smithi066 ceph-mon[46830]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:33.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:33 smithi079 ceph-mon[46738]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:33.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:33 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:33.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:33 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:33.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:33 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:33.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:33 smithi062 ceph-mon[42571]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:33.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:33 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:33.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:33 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:33.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:33 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:34.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:33 smithi066 ceph-mon[46830]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:34.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:33 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:34.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:33 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:34.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:33 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:35.566 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 8 on host 'smithi079' 2024-05-08T22:33:35.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.566+0000 7f7c2f7fe700 1 -- 172.21.15.79:0/820338576 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7c3c02cee0 con 0x7f7c280a1120 2024-05-08T22:33:35.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.568+0000 7f7c444ef700 1 -- 172.21.15.79:0/820338576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7c280a1120 msgr2=0x7f7c280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:35.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.568+0000 7f7c444ef700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7c280a1120 0x7f7c280a35e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800b2e0 tx=0x7f7c38005fb0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.569+0000 7f7c444ef700 1 -- 172.21.15.79:0/820338576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c3c0fff00 msgr2=0x7f7c3c069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:35.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.569+0000 7f7c444ef700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c3c0fff00 0x7f7c3c069d10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000e9a0 tx=0x7f7c3000ecb0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.569+0000 7f7c444ef700 1 -- 172.21.15.79:0/820338576 shutdown_connections 2024-05-08T22:33:35.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.569+0000 7f7c444ef700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7c280a1120 0x7f7c280a35e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.569+0000 7f7c444ef700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c3c101a20 0x7f7c3c19e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.569+0000 7f7c444ef700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c3c0fff00 0x7f7c3c069d10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.569+0000 7f7c444ef700 1 --2- 172.21.15.79:0/820338576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c101060 0x7f7c3c19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:35.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.569+0000 7f7c444ef700 1 -- 172.21.15.79:0/820338576 >> 172.21.15.79:0/820338576 conn(0x7f7c3c0fb4c0 msgr2=0x7f7c3c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:35.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.569+0000 7f7c444ef700 1 -- 172.21.15.79:0/820338576 shutdown_connections 2024-05-08T22:33:35.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:33:35.569+0000 7f7c444ef700 1 -- 172.21.15.79:0/820338576 wait complete. 2024-05-08T22:33:35.777 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:35 smithi079 ceph-mon[46738]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:35.777 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:35.777 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:35.777 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:35.777 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:35.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:35.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:35.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:35.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:35 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:35.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:35 smithi062 ceph-mon[42571]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:35.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:35 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:35.961 DEBUG:teuthology.orchestra.run.smithi079:osd.8> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.8.service 2024-05-08T22:33:35.963 INFO:tasks.cephadm:Deploying osd.9 on smithi062 with /dev/vg_nvme/lv_1... 2024-05-08T22:33:35.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T22:33:36.006 INFO:journalctl@ceph.osd.8.smithi079.stdout:-- Logs begin at Wed 2024-05-08 22:12:11 UTC. -- 2024-05-08T22:33:36.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:35 smithi066 ceph-mon[46830]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:36.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:36.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:36.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:36.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:36.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:36.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:36.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:36.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:35 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:36.221 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:33:37.544 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T22:33:37.544 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T22:33:37.545 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-05-08T22:33:37.545 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-05-08T22:33:37.545 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111497 s, 940 MB/s 2024-05-08T22:33:37.545 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-05-08T22:33:37.638 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:33:37 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:33:37.213+0000 7f095ede67c0 -1 osd.8 0 log_to_monitors true 2024-05-08T22:33:37.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:37 smithi062 ceph-mon[42571]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:37.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:37 smithi062 ceph-mon[42571]: from='osd.8 [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:33:37.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:37 smithi062 ceph-mon[42571]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:33:37.931 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_1 2024-05-08T22:33:37.968 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:37 smithi079 ceph-mon[46738]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:37.968 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:37 smithi079 ceph-mon[46738]: from='osd.8 [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:33:37.969 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:37 smithi079 ceph-mon[46738]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:33:38.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:37 smithi066 ceph-mon[46830]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:38.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:37 smithi066 ceph-mon[46830]: from='osd.8 [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:33:38.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:37 smithi066 ceph-mon[46830]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T22:33:38.157 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:33:38.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T22:33:38.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T22:33:38.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:38.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='osd.8 [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:33:38.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:33:38.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:38.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:38.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:38.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:38.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:38.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:38.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:38.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:38.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:38 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:39.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T22:33:39.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T22:33:39.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:39.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='osd.8 [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:33:39.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:33:39.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:39.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:39.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:39.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:39.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:39.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:39.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:39.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:39.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:38 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:39.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T22:33:39.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T22:33:39.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='osd.8 [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:33:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:33:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:38 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:39.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.266+0000 7fe37359e700 1 -- 172.21.15.62:0/4062191143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe374101f80 msgr2=0x7fe374102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.266+0000 7fe37359e700 1 --2- 172.21.15.62:0/4062191143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe374101f80 0x7fe374102400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe368009990 tx=0x7fe368009ca0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.266+0000 7fe37359e700 1 -- 172.21.15.62:0/4062191143 shutdown_connections 2024-05-08T22:33:39.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.266+0000 7fe37359e700 1 --2- 172.21.15.62:0/4062191143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374102940 0x7fe374109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.266+0000 7fe37359e700 1 --2- 172.21.15.62:0/4062191143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe374100e20 0x7fe374101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.267+0000 7fe37359e700 1 --2- 172.21.15.62:0/4062191143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe374101f80 0x7fe374102400 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.267+0000 7fe37359e700 1 -- 172.21.15.62:0/4062191143 >> 172.21.15.62:0/4062191143 conn(0x7fe3740fc3e0 msgr2=0x7fe3740fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.267+0000 7fe37359e700 1 -- 172.21.15.62:0/4062191143 shutdown_connections 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.267+0000 7fe37359e700 1 -- 172.21.15.62:0/4062191143 wait complete. 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.267+0000 7fe37359e700 1 Processor -- start 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.267+0000 7fe37359e700 1 -- start start 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.267+0000 7fe37359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe374100e20 0x7fe374196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37359e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374101f80 0x7fe374197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37359e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe374102940 0x7fe37419e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37419e810 con 0x7fe374100e20 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37359e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37419e950 con 0x7fe374101f80 2024-05-08T22:33:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37359e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37419eb10 con 0x7fe374102940 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe374100e20 0x7fe374196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe374100e20 0x7fe374196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33890/0 (socket says 172.21.15.62:33890) 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37259c700 1 -- 172.21.15.62:0/2914267227 learned_addr learned my addr 172.21.15.62:0/2914267227 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe371d9b700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374101f80 0x7fe374197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe372d9d700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe374102940 0x7fe37419e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37259c700 1 -- 172.21.15.62:0/2914267227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe374102940 msgr2=0x7fe37419e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37259c700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe374102940 0x7fe37419e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37259c700 1 -- 172.21.15.62:0/2914267227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374101f80 msgr2=0x7fe374197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37259c700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374101f80 0x7fe374197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37259c700 1 -- 172.21.15.62:0/2914267227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680095f0 con 0x7fe374100e20 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37259c700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe374100e20 0x7fe374196ce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe36400d950 tx=0x7fe36400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe3637fe700 1 -- 172.21.15.62:0/2914267227 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36400f890 con 0x7fe374100e20 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe3637fe700 1 -- 172.21.15.62:0/2914267227 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe364010460 con 0x7fe374100e20 2024-05-08T22:33:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37359e700 1 -- 172.21.15.62:0/2914267227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37419edf0 con 0x7fe374100e20 2024-05-08T22:33:39.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe3637fe700 1 -- 172.21.15.62:0/2914267227 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36400e610 con 0x7fe374100e20 2024-05-08T22:33:39.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.268+0000 7fe37359e700 1 -- 172.21.15.62:0/2914267227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37419f200 con 0x7fe374100e20 2024-05-08T22:33:39.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.269+0000 7fe37359e700 1 -- 172.21.15.62:0/2914267227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe374108250 con 0x7fe374100e20 2024-05-08T22:33:39.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.271+0000 7fe3637fe700 1 -- 172.21.15.62:0/2914267227 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe3640108d0 con 0x7fe374100e20 2024-05-08T22:33:39.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.271+0000 7fe3637fe700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe35c0a1120 0x7fe35c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:39.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.272+0000 7fe371d9b700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe35c0a1120 0x7fe35c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:39.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.272+0000 7fe3637fe700 1 -- 172.21.15.62:0/2914267227 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5549+0+0 (secure 0 0 0) 0x7fe3640cbb90 con 0x7fe374100e20 2024-05-08T22:33:39.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.272+0000 7fe371d9b700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe35c0a1120 0x7fe35c0a35e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe3741981a0 tx=0x7fe368003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:39.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.275+0000 7fe3637fe700 1 -- 172.21.15.62:0/2914267227 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe364095df0 con 0x7fe374100e20 2024-05-08T22:33:39.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:39.487+0000 7fe37359e700 1 -- 172.21.15.62:0/2914267227 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe37402cee0 con 0x7fe35c0a1120 2024-05-08T22:33:39.888 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:33:39 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:33:39.440+0000 7f0953c49700 -1 osd.8 0 waiting for initial osdmap 2024-05-08T22:33:39.889 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:33:39 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:33:39.459+0000 7f094c237700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:39.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:39 smithi079 ceph-mon[46738]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:39.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:39 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi079 2024-05-08T22:33:39.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:39 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi079 to 5704M 2024-05-08T22:33:39.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:39 smithi079 ceph-mon[46738]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:33:39.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:39 smithi079 ceph-mon[46738]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T22:33:39.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:39 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:39.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:39 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:39.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:39 smithi079 ceph-mon[46738]: from='osd.8 ' entity='osd.8' 2024-05-08T22:33:39.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:39 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:39.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:39 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:39.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:39 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:39.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:39 smithi062 ceph-mon[42571]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:39.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:39 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi079 2024-05-08T22:33:39.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:39 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi079 to 5704M 2024-05-08T22:33:39.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:39 smithi062 ceph-mon[42571]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:33:39.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:39 smithi062 ceph-mon[42571]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T22:33:39.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:39 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:39.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:39 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:39.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:39 smithi062 ceph-mon[42571]: from='osd.8 ' entity='osd.8' 2024-05-08T22:33:39.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:39 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:39.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:39 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:39.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:39 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:40.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:39 smithi066 ceph-mon[46830]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-08T22:33:40.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:39 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi079 2024-05-08T22:33:40.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:39 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi079 to 5704M 2024-05-08T22:33:40.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:39 smithi066 ceph-mon[46830]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:33:40.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:39 smithi066 ceph-mon[46830]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T22:33:40.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:39 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:40.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:39 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:40.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:39 smithi066 ceph-mon[46830]: from='osd.8 ' entity='osd.8' 2024-05-08T22:33:40.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:39 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:40.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:39 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:40.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:39 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:40.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:40 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:33:40.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:40 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:33:40.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:40 smithi062 ceph-mon[42571]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:40.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:40 smithi062 ceph-mon[42571]: osd.8 [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040] boot 2024-05-08T22:33:40.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:40 smithi062 ceph-mon[42571]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T22:33:40.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:40 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:40.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:40 smithi062 ceph-mon[42571]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T22:33:41.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:40 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:33:41.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:40 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:33:41.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:40 smithi066 ceph-mon[46830]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:41.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:40 smithi066 ceph-mon[46830]: osd.8 [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040] boot 2024-05-08T22:33:41.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:40 smithi066 ceph-mon[46830]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T22:33:41.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:40 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:41.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:40 smithi066 ceph-mon[46830]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T22:33:41.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:40 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:33:41.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:40 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:33:41.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:40 smithi079 ceph-mon[46738]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:41.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:40 smithi079 ceph-mon[46738]: osd.8 [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040] boot 2024-05-08T22:33:41.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:40 smithi079 ceph-mon[46738]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T22:33:41.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:40 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T22:33:41.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:40 smithi079 ceph-mon[46738]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T22:33:41.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:41 smithi062 ceph-mon[42571]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:41.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:41 smithi062 ceph-mon[42571]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T22:33:41.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:41 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/27556329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1811353e-323e-4f18-8c1b-5121e896aa68"}]: dispatch 2024-05-08T22:33:41.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:41 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/27556329' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1811353e-323e-4f18-8c1b-5121e896aa68"}]': finished 2024-05-08T22:33:41.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:41 smithi062 ceph-mon[42571]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T22:33:41.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:41 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:42.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:41 smithi066 ceph-mon[46830]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:42.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:41 smithi066 ceph-mon[46830]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T22:33:42.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:41 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/27556329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1811353e-323e-4f18-8c1b-5121e896aa68"}]: dispatch 2024-05-08T22:33:42.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:41 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/27556329' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1811353e-323e-4f18-8c1b-5121e896aa68"}]': finished 2024-05-08T22:33:42.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:41 smithi066 ceph-mon[46830]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T22:33:42.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:41 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:42.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:41 smithi079 ceph-mon[46738]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:42.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:41 smithi079 ceph-mon[46738]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T22:33:42.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:41 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/27556329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1811353e-323e-4f18-8c1b-5121e896aa68"}]: dispatch 2024-05-08T22:33:42.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:41 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/27556329' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1811353e-323e-4f18-8c1b-5121e896aa68"}]': finished 2024-05-08T22:33:42.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:41 smithi079 ceph-mon[46738]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T22:33:42.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:41 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:43.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:42 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/759509112' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:33:43.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:42 smithi066 ceph-mon[46830]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T22:33:43.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:42 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:43.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:42 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/759509112' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:33:43.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:42 smithi079 ceph-mon[46738]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T22:33:43.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:42 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:43.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:42 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/759509112' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:33:43.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:42 smithi062 ceph-mon[42571]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T22:33:43.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:42 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:44.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:43 smithi066 ceph-mon[46830]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail; 173 KiB/s, 0 objects/s recovering 2024-05-08T22:33:44.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:43 smithi079 ceph-mon[46738]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail; 173 KiB/s, 0 objects/s recovering 2024-05-08T22:33:44.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:43 smithi062 ceph-mon[42571]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail; 173 KiB/s, 0 objects/s recovering 2024-05-08T22:33:46.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:45 smithi062 ceph-mon[42571]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T22:33:46.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:45 smithi066 ceph-mon[46830]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T22:33:46.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:45 smithi079 ceph-mon[46738]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T22:33:47.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:47 smithi062 ceph-mon[42571]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-08T22:33:47.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T22:33:47.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:48.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:47 smithi066 ceph-mon[46830]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-08T22:33:48.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T22:33:48.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:48.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:47 smithi079 ceph-mon[46738]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-08T22:33:48.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T22:33:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:48.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:48 smithi062 ceph-mon[42571]: Deploying daemon osd.9 on smithi062 2024-05-08T22:33:49.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:48 smithi066 ceph-mon[46830]: Deploying daemon osd.9 on smithi062 2024-05-08T22:33:49.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:48 smithi079 ceph-mon[46738]: Deploying daemon osd.9 on smithi062 2024-05-08T22:33:49.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:49 smithi062 ceph-mon[42571]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-08T22:33:50.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:49 smithi066 ceph-mon[46830]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-08T22:33:50.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:49 smithi079 ceph-mon[46738]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-08T22:33:51.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:51 smithi062 ceph-mon[42571]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T22:33:51.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:51.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:51.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:52.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:51 smithi066 ceph-mon[46830]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T22:33:52.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:52.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:52.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:52.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:51 smithi079 ceph-mon[46738]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T22:33:52.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:52.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:52.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:53.689 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 9 on host 'smithi062' 2024-05-08T22:33:53.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.688+0000 7fe3637fe700 1 -- 172.21.15.62:0/2914267227 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe37402cee0 con 0x7fe35c0a1120 2024-05-08T22:33:53.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 -- 172.21.15.62:0/2914267227 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe35c0a1120 msgr2=0x7fe35c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:53.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe35c0a1120 0x7fe35c0a35e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe3741981a0 tx=0x7fe368003790 comp rx=0 tx=0).stop 2024-05-08T22:33:53.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 -- 172.21.15.62:0/2914267227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe374100e20 msgr2=0x7fe374196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:53.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe374100e20 0x7fe374196ce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe36400d950 tx=0x7fe36400dc60 comp rx=0 tx=0).stop 2024-05-08T22:33:53.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 -- 172.21.15.62:0/2914267227 shutdown_connections 2024-05-08T22:33:53.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe374102940 0x7fe37419e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:53.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe374100e20 0x7fe374196ce0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:53.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe35c0a1120 0x7fe35c0a35e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:53.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 --2- 172.21.15.62:0/2914267227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374101f80 0x7fe374197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:53.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 -- 172.21.15.62:0/2914267227 >> 172.21.15.62:0/2914267227 conn(0x7fe3740fc3e0 msgr2=0x7fe374105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:53.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 -- 172.21.15.62:0/2914267227 shutdown_connections 2024-05-08T22:33:53.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:33:53.690+0000 7fe37359e700 1 -- 172.21.15.62:0/2914267227 wait complete. 2024-05-08T22:33:54.092 DEBUG:teuthology.orchestra.run.smithi062:osd.9> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.9.service 2024-05-08T22:33:54.094 INFO:tasks.cephadm:Deploying osd.10 on smithi066 with /dev/vg_nvme/lv_1... 2024-05-08T22:33:54.094 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T22:33:54.129 INFO:journalctl@ceph.osd.9.smithi062.stdout:-- Logs begin at Wed 2024-05-08 22:12:07 UTC. -- 2024-05-08T22:33:54.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:53 smithi062 ceph-mon[42571]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:54.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:54.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:53 smithi066 ceph-mon[46830]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:54.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:54.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:54.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:33:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:53 smithi079 ceph-mon[46738]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:54.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:33:55.615 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T22:33:55.615 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T22:33:55.615 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-05-08T22:33:55.615 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-05-08T22:33:55.615 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011074 s, 947 MB/s 2024-05-08T22:33:55.615 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-05-08T22:33:55.671 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:33:55 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:33:55.188+0000 7f78413007c0 -1 osd.9 0 log_to_monitors true 2024-05-08T22:33:55.988 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_1 2024-05-08T22:33:56.210 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:55 smithi062 ceph-mon[42571]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:56.210 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:55 smithi062 ceph-mon[42571]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T22:33:56.216 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:33:56.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:55 smithi066 ceph-mon[46830]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:56.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:55 smithi066 ceph-mon[46830]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T22:33:56.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:55 smithi079 ceph-mon[46738]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:56.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:55 smithi079 ceph-mon[46738]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T22:33:57.154 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T22:33:57.154 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T22:33:57.154 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T22:33:57.155 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:57.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.290+0000 7fc22535f700 1 -- 172.21.15.66:0/1523955248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2200ffe70 msgr2=0x7fc220100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:57.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.290+0000 7fc22535f700 1 --2- 172.21.15.66:0/1523955248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2200ffe70 0x7fc220100290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc210009990 tx=0x7fc210009ca0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.290+0000 7fc22535f700 1 -- 172.21.15.66:0/1523955248 shutdown_connections 2024-05-08T22:33:57.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.290+0000 7fc22535f700 1 --2- 172.21.15.66:0/1523955248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc220101990 0x7fc2201082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.290+0000 7fc22535f700 1 --2- 172.21.15.66:0/1523955248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2200ffe70 0x7fc220100290 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.290+0000 7fc22535f700 1 --2- 172.21.15.66:0/1523955248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc220100fd0 0x7fc220101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.290+0000 7fc22535f700 1 -- 172.21.15.66:0/1523955248 >> 172.21.15.66:0/1523955248 conn(0x7fc2200fb450 msgr2=0x7fc2200fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:33:57.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.290+0000 7fc22535f700 1 -- 172.21.15.66:0/1523955248 shutdown_connections 2024-05-08T22:33:57.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.290+0000 7fc22535f700 1 -- 172.21.15.66:0/1523955248 wait complete. 2024-05-08T22:33:57.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.291+0000 7fc22535f700 1 Processor -- start 2024-05-08T22:33:57.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.291+0000 7fc22535f700 1 -- start start 2024-05-08T22:33:57.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.291+0000 7fc22535f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2200ffe70 0x7fc220103d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:57.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.291+0000 7fc22535f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc220100fd0 0x7fc220102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:57.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.291+0000 7fc22535f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc220101990 0x7fc220102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:57.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.291+0000 7fc22535f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc220103060 con 0x7fc2200ffe70 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.291+0000 7fc22535f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2201031d0 con 0x7fc220101990 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.291+0000 7fc22535f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2201a0f50 con 0x7fc220100fd0 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc220101990 0x7fc220102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2200ffe70 0x7fc220103d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc220101990 0x7fc220102980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48602/0 (socket says 172.21.15.66:48602) 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21f7fe700 1 -- 172.21.15.66:0/4051473821 learned_addr learned my addr 172.21.15.66:0/4051473821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21effd700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2200ffe70 0x7fc220103d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.66:42266/0 (socket says 172.21.15.66:42266) 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21e7fc700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc220100fd0 0x7fc220102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21f7fe700 1 -- 172.21.15.66:0/4051473821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc220100fd0 msgr2=0x7fc220102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21f7fe700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc220100fd0 0x7fc220102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21f7fe700 1 -- 172.21.15.66:0/4051473821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2200ffe70 msgr2=0x7fc220103d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:33:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21f7fe700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2200ffe70 0x7fc220103d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:33:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21f7fe700 1 -- 172.21.15.66:0/4051473821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2100095f0 con 0x7fc220101990 2024-05-08T22:33:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21effd700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2200ffe70 0x7fc220103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:33:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc21f7fe700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc220101990 0x7fc220102980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc21800ef30 tx=0x7fc21800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc217fff700 1 -- 172.21.15.66:0/4051473821 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218018070 con 0x7fc220101990 2024-05-08T22:33:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc217fff700 1 -- 172.21.15.66:0/4051473821 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc21800f040 con 0x7fc220101990 2024-05-08T22:33:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc22535f700 1 -- 172.21.15.66:0/4051473821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2201a1150 con 0x7fc220101990 2024-05-08T22:33:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc217fff700 1 -- 172.21.15.66:0/4051473821 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21800cd00 con 0x7fc220101990 2024-05-08T22:33:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.292+0000 7fc22535f700 1 -- 172.21.15.66:0/4051473821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2201a15d0 con 0x7fc220101990 2024-05-08T22:33:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.293+0000 7fc22535f700 1 -- 172.21.15.66:0/4051473821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22004ec60 con 0x7fc220101990 2024-05-08T22:33:57.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.295+0000 7fc217fff700 1 -- 172.21.15.66:0/4051473821 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc218003a00 con 0x7fc220101990 2024-05-08T22:33:57.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.296+0000 7fc217fff700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc20c0a1120 0x7fc20c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:33:57.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.296+0000 7fc217fff700 1 -- 172.21.15.66:0/4051473821 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5981+0+0 (secure 0 0 0) 0x7fc2180cc4f0 con 0x7fc220101990 2024-05-08T22:33:57.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.296+0000 7fc21effd700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc20c0a1120 0x7fc20c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:33:57.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.296+0000 7fc21effd700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc20c0a1120 0x7fc20c0a35e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc21000b2e0 tx=0x7fc210005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:33:57.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.299+0000 7fc217fff700 1 -- 172.21.15.66:0/4051473821 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc218096590 con 0x7fc220101990 2024-05-08T22:33:57.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T22:33:57.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T22:33:57.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:33:57.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T22:33:57.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:57.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T22:33:57.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:33:57.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:33:57.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-05-08T22:33:57.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T22:33:57.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:33:57.511+0000 7fc22535f700 1 -- 172.21.15.66:0/4051473821 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc220069cd0 con 0x7fc20c0a1120 2024-05-08T22:33:58.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:57 smithi062 ceph-mon[42571]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:58.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:57 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi062 2024-05-08T22:33:58.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:57 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi062 to 4278M 2024-05-08T22:33:58.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:58.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:57 smithi062 ceph-mon[42571]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:58.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:58.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:58.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:58.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:57 smithi062 ceph-mon[42571]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' 2024-05-08T22:33:58.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:58.173 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:33:57 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:33:57.800+0000 7f7836163700 -1 osd.9 0 waiting for initial osdmap 2024-05-08T22:33:58.173 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:33:57 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:33:57.820+0000 7f782e751700 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:33:58.192 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:57 smithi066 ceph-mon[46830]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:58.192 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:57 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi062 2024-05-08T22:33:58.192 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:57 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi062 to 4278M 2024-05-08T22:33:58.192 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:58.192 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:57 smithi066 ceph-mon[46830]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:58.192 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:58.192 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:58.192 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:58.192 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:57 smithi066 ceph-mon[46830]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' 2024-05-08T22:33:58.192 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:58.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:57 smithi079 ceph-mon[46738]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:58.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:57 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi062 2024-05-08T22:33:58.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:57 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi062 to 4278M 2024-05-08T22:33:58.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:58.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:57 smithi079 ceph-mon[46738]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:33:58.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:33:58.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:33:58.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:33:58.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:57 smithi079 ceph-mon[46738]: from='osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351]' entity='osd.9' 2024-05-08T22:33:58.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:58 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:33:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:58 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:33:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:58 smithi066 ceph-mon[46830]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:58 smithi066 ceph-mon[46830]: osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351] boot 2024-05-08T22:33:59.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:58 smithi066 ceph-mon[46830]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T22:33:59.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:58 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:58 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:33:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:58 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:33:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:58 smithi079 ceph-mon[46738]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:58 smithi079 ceph-mon[46738]: osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351] boot 2024-05-08T22:33:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:58 smithi079 ceph-mon[46738]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T22:33:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:58 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:33:59.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:58 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:33:59.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:58 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:33:59.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:58 smithi062 ceph-mon[42571]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-08T22:33:59.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:58 smithi062 ceph-mon[42571]: osd.9 [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351] boot 2024-05-08T22:33:59.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:58 smithi062 ceph-mon[42571]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T22:33:59.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:58 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T22:34:00.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:59 smithi066 ceph-mon[46830]: from='client.? 172.21.15.66:0/875711897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1769ba4-8dd8-4fb2-a596-e37d465a58d9"}]: dispatch 2024-05-08T22:34:00.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:59 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1769ba4-8dd8-4fb2-a596-e37d465a58d9"}]: dispatch 2024-05-08T22:34:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:59 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1769ba4-8dd8-4fb2-a596-e37d465a58d9"}]': finished 2024-05-08T22:34:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:59 smithi066 ceph-mon[46830]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T22:34:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:33:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:00.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:59 smithi079 ceph-mon[46738]: from='client.? 172.21.15.66:0/875711897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1769ba4-8dd8-4fb2-a596-e37d465a58d9"}]: dispatch 2024-05-08T22:34:00.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:59 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1769ba4-8dd8-4fb2-a596-e37d465a58d9"}]: dispatch 2024-05-08T22:34:00.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:59 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1769ba4-8dd8-4fb2-a596-e37d465a58d9"}]': finished 2024-05-08T22:34:00.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:59 smithi079 ceph-mon[46738]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T22:34:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:33:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:00.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:59 smithi062 ceph-mon[42571]: from='client.? 172.21.15.66:0/875711897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1769ba4-8dd8-4fb2-a596-e37d465a58d9"}]: dispatch 2024-05-08T22:34:00.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:59 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1769ba4-8dd8-4fb2-a596-e37d465a58d9"}]: dispatch 2024-05-08T22:34:00.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:59 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1769ba4-8dd8-4fb2-a596-e37d465a58d9"}]': finished 2024-05-08T22:34:00.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:59 smithi062 ceph-mon[42571]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T22:34:00.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:33:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:01.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:00 smithi066 ceph-mon[46830]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:01.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:00 smithi066 ceph-mon[46830]: from='client.? 172.21.15.66:0/1842888413' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:34:01.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:00 smithi079 ceph-mon[46738]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:01.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:00 smithi079 ceph-mon[46738]: from='client.? 172.21.15.66:0/1842888413' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:34:01.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:00 smithi062 ceph-mon[42571]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:01.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:00 smithi062 ceph-mon[42571]: from='client.? 172.21.15.66:0/1842888413' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:34:03.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:03 smithi066 ceph-mon[46830]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:03.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:03 smithi079 ceph-mon[46738]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:03.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:03 smithi062 ceph-mon[42571]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:05.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:05 smithi066 ceph-mon[46830]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:05.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:05 smithi079 ceph-mon[46738]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:05.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:05 smithi062 ceph-mon[42571]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:06.149 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T22:34:06.149 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:06.149 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:06 smithi066 ceph-mon[46830]: Deploying daemon osd.10 on smithi066 2024-05-08T22:34:06.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T22:34:06.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:06.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:06 smithi079 ceph-mon[46738]: Deploying daemon osd.10 on smithi066 2024-05-08T22:34:06.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T22:34:06.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:06.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:06 smithi062 ceph-mon[42571]: Deploying daemon osd.10 on smithi066 2024-05-08T22:34:07.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:07 smithi066 ceph-mon[46830]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:07.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:07 smithi079 ceph-mon[46738]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:07.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:07 smithi062 ceph-mon[42571]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:09.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:09 smithi066 ceph-mon[46830]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:09.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:09 smithi079 ceph-mon[46738]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:09.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:09 smithi062 ceph-mon[42571]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:10.573 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:10 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:10.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:10 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:10.574 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:10 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:10.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:10 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:10.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:10 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:10.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:10 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:10.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:10 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:10.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:10 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:10.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:10 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:11.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.314+0000 7fc217fff700 1 -- 172.21.15.66:0/4051473821 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc220069cd0 con 0x7fc20c0a1120 2024-05-08T22:34:11.315 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 10 on host 'smithi066' 2024-05-08T22:34:11.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 -- 172.21.15.66:0/4051473821 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc20c0a1120 msgr2=0x7fc20c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc20c0a1120 0x7fc20c0a35e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc21000b2e0 tx=0x7fc210005fd0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 -- 172.21.15.66:0/4051473821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc220101990 msgr2=0x7fc220102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:11.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc220101990 0x7fc220102980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc21800ef30 tx=0x7fc21800c3a0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 -- 172.21.15.66:0/4051473821 shutdown_connections 2024-05-08T22:34:11.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc20c0a1120 0x7fc20c0a35e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc220101990 0x7fc220102980 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2200ffe70 0x7fc220103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 --2- 172.21.15.66:0/4051473821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc220100fd0 0x7fc220102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:11.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 -- 172.21.15.66:0/4051473821 >> 172.21.15.66:0/4051473821 conn(0x7fc2200fb450 msgr2=0x7fc2200695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:11.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 -- 172.21.15.66:0/4051473821 shutdown_connections 2024-05-08T22:34:11.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-08T22:34:11.316+0000 7fc22535f700 1 -- 172.21.15.66:0/4051473821 wait complete. 2024-05-08T22:34:11.683 DEBUG:teuthology.orchestra.run.smithi066:osd.10> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.10.service 2024-05-08T22:34:11.686 INFO:tasks.cephadm:Deploying osd.11 on smithi079 with /dev/vg_nvme/lv_1... 2024-05-08T22:34:11.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T22:34:11.718 INFO:journalctl@ceph.osd.10.smithi066.stdout:-- Logs begin at Wed 2024-05-08 22:13:17 UTC. -- 2024-05-08T22:34:11.919 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:34:12.027 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:11 smithi079 ceph-mon[46738]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:12.027 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.027 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:12.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:12.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:12.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:11 smithi066 ceph-mon[46830]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:12.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:12.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:11 smithi062 ceph-mon[42571]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:12.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:12.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:13.237 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T22:34:13.237 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T22:34:13.237 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-05-08T22:34:13.237 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-05-08T22:34:13.237 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124962 s, 839 MB/s 2024-05-08T22:34:13.237 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-05-08T22:34:13.573 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:34:13 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:34:13.289+0000 7fe35e1247c0 -1 osd.10 0 log_to_monitors true 2024-05-08T22:34:13.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_1 2024-05-08T22:34:13.860 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:34:13.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:13 smithi079 ceph-mon[46738]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:13.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:13 smithi079 ceph-mon[46738]: from='osd.10 [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:34:13.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:13 smithi079 ceph-mon[46738]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:34:14.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:13 smithi066 ceph-mon[46830]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:14.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:13 smithi066 ceph-mon[46830]: from='osd.10 [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:34:14.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:13 smithi066 ceph-mon[46830]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:34:14.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:13 smithi062 ceph-mon[42571]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:14.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:13 smithi062 ceph-mon[42571]: from='osd.10 [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:34:14.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:13 smithi062 ceph-mon[42571]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T22:34:14.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.856+0000 7fb5e895e700 1 -- 172.21.15.79:0/1499291109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5e41043e0 msgr2=0x7fb5e41067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.856+0000 7fb5e895e700 1 --2- 172.21.15.79:0/1499291109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5e41043e0 0x7fb5e41067d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009990 tx=0x7fb5d8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.856+0000 7fb5e895e700 1 -- 172.21.15.79:0/1499291109 shutdown_connections 2024-05-08T22:34:14.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.856+0000 7fb5e895e700 1 --2- 172.21.15.79:0/1499291109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e4106d10 0x7fb5e4109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.856+0000 7fb5e895e700 1 --2- 172.21.15.79:0/1499291109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e4101ab0 0x7fb5e4103ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.856+0000 7fb5e895e700 1 --2- 172.21.15.79:0/1499291109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5e41043e0 0x7fb5e41067d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.856+0000 7fb5e895e700 1 -- 172.21.15.79:0/1499291109 >> 172.21.15.79:0/1499291109 conn(0x7fb5e40fb4c0 msgr2=0x7fb5e40fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:14.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.856+0000 7fb5e895e700 1 -- 172.21.15.79:0/1499291109 shutdown_connections 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.856+0000 7fb5e895e700 1 -- 172.21.15.79:0/1499291109 wait complete. 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e895e700 1 Processor -- start 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e895e700 1 -- start start 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e895e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e4101ab0 0x7fb5e4196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e895e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5e41043e0 0x7fb5e4197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e895e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e4106d10 0x7fb5e419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e895e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e419ea40 con 0x7fb5e4106d10 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e895e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e419eb80 con 0x7fb5e4101ab0 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e895e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e419ed10 con 0x7fb5e41043e0 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5e41043e0 0x7fb5e4197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5e41043e0 0x7fb5e4197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53248/0 (socket says 172.21.15.79:53248) 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e1d9b700 1 -- 172.21.15.79:0/1507677566 learned_addr learned my addr 172.21.15.79:0/1507677566 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e259c700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e4101ab0 0x7fb5e4196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e2d9d700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e4106d10 0x7fb5e419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e259c700 1 -- 172.21.15.79:0/1507677566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5e41043e0 msgr2=0x7fb5e4197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e259c700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5e41043e0 0x7fb5e4197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e259c700 1 -- 172.21.15.79:0/1507677566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e4106d10 msgr2=0x7fb5e419e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:14.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e259c700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e4106d10 0x7fb5e419e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:14.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e259c700 1 -- 172.21.15.79:0/1507677566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d80095f0 con 0x7fb5e4101ab0 2024-05-08T22:34:14.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e1d9b700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5e41043e0 0x7fb5e4197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:14.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.857+0000 7fb5e259c700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e4101ab0 0x7fb5e4196f00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400d750 tx=0x7fb5d400da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:14.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.875+0000 7fb5d37fe700 1 -- 172.21.15.79:0/1507677566 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4004020 con 0x7fb5e4101ab0 2024-05-08T22:34:14.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.875+0000 7fb5d37fe700 1 -- 172.21.15.79:0/1507677566 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5d4004830 con 0x7fb5e4101ab0 2024-05-08T22:34:14.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.875+0000 7fb5e895e700 1 -- 172.21.15.79:0/1507677566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e419eff0 con 0x7fb5e4101ab0 2024-05-08T22:34:14.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.875+0000 7fb5d37fe700 1 -- 172.21.15.79:0/1507677566 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40054f0 con 0x7fb5e4101ab0 2024-05-08T22:34:14.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.875+0000 7fb5e895e700 1 -- 172.21.15.79:0/1507677566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e419f410 con 0x7fb5e4101ab0 2024-05-08T22:34:14.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.876+0000 7fb5e895e700 1 -- 172.21.15.79:0/1507677566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e40fd040 con 0x7fb5e4101ab0 2024-05-08T22:34:14.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.878+0000 7fb5d37fe700 1 -- 172.21.15.79:0/1507677566 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb5d40049a0 con 0x7fb5e4101ab0 2024-05-08T22:34:14.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.878+0000 7fb5d37fe700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb5cc0a11a0 0x7fb5cc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:14.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.878+0000 7fb5e1d9b700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb5cc0a11a0 0x7fb5cc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:14.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.879+0000 7fb5e1d9b700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb5cc0a11a0 0x7fb5cc0a3660 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800b2e0 tx=0x7fb5d800b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:14.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.899+0000 7fb5d37fe700 1 -- 172.21.15.79:0/1507677566 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb5d4099fe0 con 0x7fb5e4101ab0 2024-05-08T22:34:14.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:14.904+0000 7fb5d37fe700 1 -- 172.21.15.79:0/1507677566 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb5d4095c20 con 0x7fb5e4101ab0 2024-05-08T22:34:15.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T22:34:15.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='osd.10 [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi066 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi066 to 4278M 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:15.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:15.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:15.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:14 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:15.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:15.101+0000 7fb5e895e700 1 -- 172.21.15.79:0/1507677566 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5e402cee0 con 0x7fb5cc0a11a0 2024-05-08T22:34:15.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T22:34:15.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T22:34:15.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:15.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='osd.10 [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:34:15.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:34:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi066 2024-05-08T22:34:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi066 to 4278M 2024-05-08T22:34:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:15.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:14 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:15.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T22:34:15.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T22:34:15.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='osd.10 [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi066 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi066 to 4278M 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:14 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:16.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:34:15 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:34:15.598+0000 7fe352f87700 -1 osd.10 0 waiting for initial osdmap 2024-05-08T22:34:16.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:34:15 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:34:15.619+0000 7fe34b575700 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:34:16.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:15 smithi066 ceph-mon[46830]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:16.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:15 smithi066 ceph-mon[46830]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:34:16.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:15 smithi066 ceph-mon[46830]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T22:34:16.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:15 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:16.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:15 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:16.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:15 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:34:16.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:15 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:34:16.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:15 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:16.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:15 smithi066 ceph-mon[46830]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T22:34:16.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:15 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:16.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:15 smithi066 ceph-mon[46830]: from='osd.10 ' entity='osd.10' 2024-05-08T22:34:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:15 smithi079 ceph-mon[46738]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:15 smithi079 ceph-mon[46738]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:34:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:15 smithi079 ceph-mon[46738]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T22:34:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:15 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:15 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:15 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:34:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:15 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:34:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:15 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:15 smithi079 ceph-mon[46738]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T22:34:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:15 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:15 smithi079 ceph-mon[46738]: from='osd.10 ' entity='osd.10' 2024-05-08T22:34:16.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:15 smithi062 ceph-mon[42571]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:16.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:15 smithi062 ceph-mon[42571]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-08T22:34:16.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:15 smithi062 ceph-mon[42571]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T22:34:16.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:15 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:16.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:15 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:16.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:15 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T22:34:16.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:15 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T22:34:16.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:15 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:16.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:15 smithi062 ceph-mon[42571]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T22:34:16.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:15 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:16.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:15 smithi062 ceph-mon[42571]: from='osd.10 ' entity='osd.10' 2024-05-08T22:34:17.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:16 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:34:17.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:16 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:34:17.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:16 smithi066 ceph-mon[46830]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:34:17.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:16 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:17.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:16 smithi066 ceph-mon[46830]: osd.10 [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413] boot 2024-05-08T22:34:17.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:16 smithi066 ceph-mon[46830]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T22:34:17.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:16 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:17.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:16 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:34:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:16 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:34:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:16 smithi079 ceph-mon[46738]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:34:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:16 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:16 smithi079 ceph-mon[46738]: osd.10 [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413] boot 2024-05-08T22:34:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:16 smithi079 ceph-mon[46738]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T22:34:17.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:16 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:17.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:16 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:34:17.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:16 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:34:17.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:16 smithi062 ceph-mon[42571]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:34:17.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:16 smithi062 ceph-mon[42571]: osd.10 [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413] boot 2024-05-08T22:34:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:16 smithi062 ceph-mon[42571]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T22:34:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:16 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T22:34:18.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:17 smithi066 ceph-mon[46830]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:18.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:17 smithi066 ceph-mon[46830]: from='client.? 172.21.15.79:0/2811548140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d7d0579-5d22-46e8-a43b-714675f3dd80"}]: dispatch 2024-05-08T22:34:18.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:17 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d7d0579-5d22-46e8-a43b-714675f3dd80"}]: dispatch 2024-05-08T22:34:18.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:17 smithi066 ceph-mon[46830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d7d0579-5d22-46e8-a43b-714675f3dd80"}]': finished 2024-05-08T22:34:18.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:17 smithi066 ceph-mon[46830]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T22:34:18.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:17 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:18.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:17 smithi066 ceph-mon[46830]: from='client.? 172.21.15.79:0/3939157169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:34:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:17 smithi079 ceph-mon[46738]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:17 smithi079 ceph-mon[46738]: from='client.? 172.21.15.79:0/2811548140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d7d0579-5d22-46e8-a43b-714675f3dd80"}]: dispatch 2024-05-08T22:34:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:17 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d7d0579-5d22-46e8-a43b-714675f3dd80"}]: dispatch 2024-05-08T22:34:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:17 smithi079 ceph-mon[46738]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d7d0579-5d22-46e8-a43b-714675f3dd80"}]': finished 2024-05-08T22:34:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:17 smithi079 ceph-mon[46738]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T22:34:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:17 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:17 smithi079 ceph-mon[46738]: from='client.? 172.21.15.79:0/3939157169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:34:18.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:17 smithi062 ceph-mon[42571]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-08T22:34:18.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:17 smithi062 ceph-mon[42571]: from='client.? 172.21.15.79:0/2811548140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d7d0579-5d22-46e8-a43b-714675f3dd80"}]: dispatch 2024-05-08T22:34:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:17 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d7d0579-5d22-46e8-a43b-714675f3dd80"}]: dispatch 2024-05-08T22:34:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:17 smithi062 ceph-mon[42571]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d7d0579-5d22-46e8-a43b-714675f3dd80"}]': finished 2024-05-08T22:34:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:17 smithi062 ceph-mon[42571]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T22:34:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:17 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:18.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:17 smithi062 ceph-mon[42571]: from='client.? 172.21.15.79:0/3939157169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T22:34:19.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:19 smithi066 ceph-mon[46830]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:19.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:19 smithi066 ceph-mon[46830]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T22:34:19.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:19 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:19.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:19 smithi079 ceph-mon[46738]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:19.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:19 smithi079 ceph-mon[46738]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T22:34:19.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:19 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:19.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:19 smithi062 ceph-mon[42571]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:19.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:19 smithi062 ceph-mon[42571]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T22:34:19.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:19 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:20.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:20 smithi066 ceph-mon[46830]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T22:34:20.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:20 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:20.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:20 smithi079 ceph-mon[46738]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T22:34:20.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:20 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:20.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:20 smithi062 ceph-mon[42571]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T22:34:20.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:20 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:21.237 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:21 smithi079 ceph-mon[46738]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:21.237 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:21 smithi079 ceph-mon[46738]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T22:34:21.237 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:21 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:21.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:21 smithi066 ceph-mon[46830]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:21.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:21 smithi066 ceph-mon[46830]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T22:34:21.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:21 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:21.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:21 smithi062 ceph-mon[42571]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:21.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:21 smithi062 ceph-mon[42571]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T22:34:21.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:21 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:23.105 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:23 smithi079 ceph-mon[46738]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T22:34:23.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:23 smithi066 ceph-mon[46830]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T22:34:23.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:23 smithi062 ceph-mon[42571]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T22:34:24.309 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:24 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T22:34:24.310 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:24 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:24.310 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:24 smithi079 ceph-mon[46738]: Deploying daemon osd.11 on smithi079 2024-05-08T22:34:24.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:24 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T22:34:24.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:24 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:24.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:24 smithi066 ceph-mon[46830]: Deploying daemon osd.11 on smithi079 2024-05-08T22:34:24.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T22:34:24.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:24 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:24.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:24 smithi062 ceph-mon[42571]: Deploying daemon osd.11 on smithi079 2024-05-08T22:34:25.252 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:25 smithi079 ceph-mon[46738]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:34:25.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:25 smithi066 ceph-mon[46830]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:34:25.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:25 smithi062 ceph-mon[42571]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T22:34:28.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:27 smithi079 ceph-mon[46738]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T22:34:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:28.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:27 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:28.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:27 smithi062 ceph-mon[42571]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T22:34:28.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:28.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:28.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:27 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:28.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:27 smithi066 ceph-mon[46830]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T22:34:28.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:28.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:28.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:27 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.304 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 11 on host 'smithi079' 2024-05-08T22:34:29.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.304+0000 7fb5d37fe700 1 -- 172.21.15.79:0/1507677566 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb5e402cee0 con 0x7fb5cc0a11a0 2024-05-08T22:34:29.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 -- 172.21.15.79:0/1507677566 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb5cc0a11a0 msgr2=0x7fb5cc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb5cc0a11a0 0x7fb5cc0a3660 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800b2e0 tx=0x7fb5d800b750 comp rx=0 tx=0).stop 2024-05-08T22:34:29.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 -- 172.21.15.79:0/1507677566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e4101ab0 msgr2=0x7fb5e4196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:29.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e4101ab0 0x7fb5e4196f00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400d750 tx=0x7fb5d400da60 comp rx=0 tx=0).stop 2024-05-08T22:34:29.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 -- 172.21.15.79:0/1507677566 shutdown_connections 2024-05-08T22:34:29.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e4106d10 0x7fb5e419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb5cc0a11a0 0x7fb5cc0a3660 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e4101ab0 0x7fb5e4196f00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 --2- 172.21.15.79:0/1507677566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5e41043e0 0x7fb5e4197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:29.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 -- 172.21.15.79:0/1507677566 >> 172.21.15.79:0/1507677566 conn(0x7fb5e40fb4c0 msgr2=0x7fb5e41025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:29.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 -- 172.21.15.79:0/1507677566 shutdown_connections 2024-05-08T22:34:29.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:29.307+0000 7fb5e895e700 1 -- 172.21.15.79:0/1507677566 wait complete. 2024-05-08T22:34:29.698 DEBUG:teuthology.orchestra.run.smithi079:osd.11> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.11.service 2024-05-08T22:34:29.701 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-08T22:34:29.701 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T22:34:29.729 INFO:journalctl@ceph.osd.11.smithi079.stdout:-- Logs begin at Wed 2024-05-08 22:12:11 UTC. -- 2024-05-08T22:34:29.785 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:29 smithi079 ceph-mon[46738]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T22:34:29.785 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:29.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:29.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:29 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:29.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:29 smithi066 ceph-mon[46830]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T22:34:29.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:29.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:29.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:29 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:29.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:29 smithi062 ceph-mon[42571]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T22:34:29.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:29.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:29.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:29.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:29 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:29.946 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:34:30.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.939+0000 7efc7fd59700 1 -- 172.21.15.62:0/326764873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc780fff00 msgr2=0x7efc78100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:30.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.939+0000 7efc7fd59700 1 --2- 172.21.15.62:0/326764873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc780fff00 0x7efc78100320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efc74009990 tx=0x7efc74009ca0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.939+0000 7efc7fd59700 1 -- 172.21.15.62:0/326764873 shutdown_connections 2024-05-08T22:34:30.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.939+0000 7efc7fd59700 1 --2- 172.21.15.62:0/326764873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc78101a20 0x7efc78108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.939+0000 7efc7fd59700 1 --2- 172.21.15.62:0/326764873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc780fff00 0x7efc78100320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.939+0000 7efc7fd59700 1 --2- 172.21.15.62:0/326764873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc78101060 0x7efc781014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.939+0000 7efc7fd59700 1 -- 172.21.15.62:0/326764873 >> 172.21.15.62:0/326764873 conn(0x7efc780fb4a0 msgr2=0x7efc780fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.939+0000 7efc7fd59700 1 -- 172.21.15.62:0/326764873 shutdown_connections 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.939+0000 7efc7fd59700 1 -- 172.21.15.62:0/326764873 wait complete. 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7fd59700 1 Processor -- start 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7fd59700 1 -- start start 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7fd59700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc780fff00 0x7efc78196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7fd59700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc78101060 0x7efc78197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7fd59700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc78101a20 0x7efc7819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7fd59700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7819ea10 con 0x7efc78101060 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7fd59700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7819eb50 con 0x7efc78101a20 2024-05-08T22:34:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7fd59700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7819ece0 con 0x7efc780fff00 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7d2f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc78101060 0x7efc78197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7d2f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc78101060 0x7efc78197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33534/0 (socket says 172.21.15.62:33534) 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7d2f4700 1 -- 172.21.15.62:0/2984097745 learned_addr learned my addr 172.21.15.62:0/2984097745 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7daf5700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc780fff00 0x7efc78196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.940+0000 7efc7e2f6700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc78101a20 0x7efc7819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.941+0000 7efc7d2f4700 1 -- 172.21.15.62:0/2984097745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc780fff00 msgr2=0x7efc78196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.941+0000 7efc7d2f4700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc780fff00 0x7efc78196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.941+0000 7efc7d2f4700 1 -- 172.21.15.62:0/2984097745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc78101a20 msgr2=0x7efc7819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.941+0000 7efc7d2f4700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc78101a20 0x7efc7819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.941+0000 7efc7d2f4700 1 -- 172.21.15.62:0/2984097745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc740095f0 con 0x7efc78101060 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.941+0000 7efc7d2f4700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc78101060 0x7efc78197460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efc6800c9e0 tx=0x7efc6800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.941+0000 7efc6effd700 1 -- 172.21.15.62:0/2984097745 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68007ac0 con 0x7efc78101060 2024-05-08T22:34:30.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.941+0000 7efc7fd59700 1 -- 172.21.15.62:0/2984097745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7819efc0 con 0x7efc78101060 2024-05-08T22:34:30.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.941+0000 7efc6effd700 1 -- 172.21.15.62:0/2984097745 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc6800ce80 con 0x7efc78101060 2024-05-08T22:34:30.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.941+0000 7efc6effd700 1 -- 172.21.15.62:0/2984097745 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68018660 con 0x7efc78101060 2024-05-08T22:34:30.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.941+0000 7efc7fd59700 1 -- 172.21.15.62:0/2984097745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7819f430 con 0x7efc78101060 2024-05-08T22:34:30.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.942+0000 7efc7fd59700 1 -- 172.21.15.62:0/2984097745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7804ec60 con 0x7efc78101060 2024-05-08T22:34:30.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.943+0000 7efc6effd700 1 -- 172.21.15.62:0/2984097745 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7efc6800f450 con 0x7efc78101060 2024-05-08T22:34:30.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.944+0000 7efc6effd700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7efc640a1120 0x7efc640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:30.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.944+0000 7efc7daf5700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7efc640a1120 0x7efc640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:30.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.944+0000 7efc6effd700 1 -- 172.21.15.62:0/2984097745 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6809+0+0 (secure 0 0 0) 0x7efc680cc570 con 0x7efc78101060 2024-05-08T22:34:30.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.944+0000 7efc7daf5700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7efc640a1120 0x7efc640a35e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efc7400b5c0 tx=0x7efc74003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:30.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:30.948+0000 7efc6effd700 1 -- 172.21.15.62:0/2984097745 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efc680962d0 con 0x7efc78101060 2024-05-08T22:34:31.138 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:34:30 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:34:30.867+0000 7faf47e867c0 -1 osd.11 0 log_to_monitors true 2024-05-08T22:34:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.153+0000 7efc7fd59700 1 -- 172.21.15.62:0/2984097745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7efc7810b8e0 con 0x7efc78101060 2024-05-08T22:34:31.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.153+0000 7efc6effd700 1 -- 172.21.15.62:0/2984097745 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) v1 ==== 74+0+133 (secure 0 0 0) 0x7efc6800fe60 con 0x7efc78101060 2024-05-08T22:34:31.153 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.155+0000 7efc7fd59700 1 -- 172.21.15.62:0/2984097745 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7efc640a1120 msgr2=0x7efc640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.155+0000 7efc7fd59700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7efc640a1120 0x7efc640a35e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efc7400b5c0 tx=0x7efc74003790 comp rx=0 tx=0).stop 2024-05-08T22:34:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.155+0000 7efc7fd59700 1 -- 172.21.15.62:0/2984097745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc78101060 msgr2=0x7efc78197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.155+0000 7efc7fd59700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc78101060 0x7efc78197460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efc6800c9e0 tx=0x7efc6800ccf0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.155+0000 7efc7fd59700 1 -- 172.21.15.62:0/2984097745 shutdown_connections 2024-05-08T22:34:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.155+0000 7efc7fd59700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7efc640a1120 0x7efc640a35e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.155+0000 7efc7fd59700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc78101a20 0x7efc7819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.155+0000 7efc7fd59700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc780fff00 0x7efc78196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.155+0000 7efc7fd59700 1 --2- 172.21.15.62:0/2984097745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc78101060 0x7efc78197460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.155+0000 7efc7fd59700 1 -- 172.21.15.62:0/2984097745 >> 172.21.15.62:0/2984097745 conn(0x7efc780fb4a0 msgr2=0x7efc780fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.156+0000 7efc7fd59700 1 -- 172.21.15.62:0/2984097745 shutdown_connections 2024-05-08T22:34:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:31.156+0000 7efc7fd59700 1 -- 172.21.15.62:0/2984097745 wait complete. 2024-05-08T22:34:31.534 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1715207656,"num_in_osds":12,"osd_in_since":1715207656,"num_remapped_pgs":0} 2024-05-08T22:34:31.635 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:31 smithi079 ceph-mon[46738]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:34:31.635 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:31 smithi079 ceph-mon[46738]: from='osd.11 [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:34:31.635 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:31 smithi079 ceph-mon[46738]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:34:31.635 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:31 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2984097745' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:34:31.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:31 smithi066 ceph-mon[46830]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:34:31.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:31 smithi066 ceph-mon[46830]: from='osd.11 [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:34:31.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:31 smithi066 ceph-mon[46830]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:34:31.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:31 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2984097745' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:34:31.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:31 smithi062 ceph-mon[42571]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T22:34:31.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:31 smithi062 ceph-mon[42571]: from='osd.11 [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:34:31.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:31 smithi062 ceph-mon[42571]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T22:34:31.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:31 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2984097745' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:34:32.535 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T22:34:32.776 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:34:32.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T22:34:32.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T22:34:32.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:32.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='osd.11 [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: Detected new or changed devices on smithi079 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: Adjusting osd_memory_target on smithi079 to 4278M 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:32.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:32 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:32.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T22:34:32.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='osd.11 [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: Detected new or changed devices on smithi079 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: Adjusting osd_memory_target on smithi079 to 4278M 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:32.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:32.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:32.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:32 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:32.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T22:34:32.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T22:34:32.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:32.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='osd.11 [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:34:32.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-05-08T22:34:32.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: Detected new or changed devices on smithi079 2024-05-08T22:34:32.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:32.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:32.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T22:34:32.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: Adjusting osd_memory_target on smithi079 to 4278M 2024-05-08T22:34:32.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:32.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:32.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:32.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:32 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:33.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:34:33 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:34:33.347+0000 7faf3cce9700 -1 osd.11 0 waiting for initial osdmap 2024-05-08T22:34:33.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:34:33 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:34:33.366+0000 7faf352d7700 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T22:34:33.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:33 smithi079 ceph-mon[46738]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:33.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:33 smithi079 ceph-mon[46738]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:34:33.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:33 smithi079 ceph-mon[46738]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T22:34:33.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:33 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:33.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:33 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:33.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:33 smithi079 ceph-mon[46738]: from='osd.11 ' entity='osd.11' 2024-05-08T22:34:33.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.782+0000 7f66177f3700 1 -- 172.21.15.62:0/3679158246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610101060 msgr2=0x7f66101014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:33.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.782+0000 7f66177f3700 1 --2- 172.21.15.62:0/3679158246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610101060 0x7f66101014e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f660c009990 tx=0x7f660c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.782+0000 7f66177f3700 1 -- 172.21.15.62:0/3679158246 shutdown_connections 2024-05-08T22:34:33.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.782+0000 7f66177f3700 1 --2- 172.21.15.62:0/3679158246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6610101a20 0x7f6610108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.782+0000 7f66177f3700 1 --2- 172.21.15.62:0/3679158246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66100fff00 0x7f6610100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.782+0000 7f66177f3700 1 --2- 172.21.15.62:0/3679158246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610101060 0x7f66101014e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.782+0000 7f66177f3700 1 -- 172.21.15.62:0/3679158246 >> 172.21.15.62:0/3679158246 conn(0x7f66100fb4a0 msgr2=0x7f66100fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:33.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.782+0000 7f66177f3700 1 -- 172.21.15.62:0/3679158246 shutdown_connections 2024-05-08T22:34:33.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.782+0000 7f66177f3700 1 -- 172.21.15.62:0/3679158246 wait complete. 2024-05-08T22:34:33.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f66177f3700 1 Processor -- start 2024-05-08T22:34:33.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f66177f3700 1 -- start start 2024-05-08T22:34:33.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f66177f3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66100fff00 0x7f6610069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:33.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f66177f3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610101060 0x7f661019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:33.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f66177f3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6610101a20 0x7f661019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:33.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f66177f3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661019e9c0 con 0x7f6610101060 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f66177f3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661019eb30 con 0x7f66100fff00 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f66177f3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661019ecf0 con 0x7f6610101a20 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f6614d8e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610101060 0x7f661019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f6614d8e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610101060 0x7f661019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33546/0 (socket says 172.21.15.62:33546) 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f6614d8e700 1 -- 172.21.15.62:0/637520911 learned_addr learned my addr 172.21.15.62:0/637520911 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f6615d90700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6610101a20 0x7f661019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f661558f700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66100fff00 0x7f6610069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f6614d8e700 1 -- 172.21.15.62:0/637520911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6610101a20 msgr2=0x7f661019e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f6614d8e700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6610101a20 0x7f661019e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f6614d8e700 1 -- 172.21.15.62:0/637520911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66100fff00 msgr2=0x7f6610069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f6614d8e700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66100fff00 0x7f6610069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f6614d8e700 1 -- 172.21.15.62:0/637520911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0095f0 con 0x7f6610101060 2024-05-08T22:34:33.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f661558f700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66100fff00 0x7f6610069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:33.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.783+0000 7f6614d8e700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610101060 0x7f661019bc50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f660c005950 tx=0x7f660c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:33.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.784+0000 7f66027fc700 1 -- 172.21.15.62:0/637520911 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c01d070 con 0x7f6610101060 2024-05-08T22:34:33.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.784+0000 7f66027fc700 1 -- 172.21.15.62:0/637520911 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660c003f90 con 0x7f6610101060 2024-05-08T22:34:33.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.784+0000 7f66177f3700 1 -- 172.21.15.62:0/637520911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661019ef70 con 0x7f6610101060 2024-05-08T22:34:33.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.784+0000 7f66027fc700 1 -- 172.21.15.62:0/637520911 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c022d40 con 0x7f6610101060 2024-05-08T22:34:33.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.784+0000 7f66177f3700 1 -- 172.21.15.62:0/637520911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661019f380 con 0x7f6610101060 2024-05-08T22:34:33.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.785+0000 7f66177f3700 1 -- 172.21.15.62:0/637520911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6610107330 con 0x7f6610101060 2024-05-08T22:34:33.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.786+0000 7f66027fc700 1 -- 172.21.15.62:0/637520911 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f660c003ad0 con 0x7f6610101060 2024-05-08T22:34:33.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.786+0000 7f66027fc700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65fc0a11a0 0x7f65fc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:33.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.786+0000 7f661558f700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65fc0a11a0 0x7f65fc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:33.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.786+0000 7f66027fc700 1 -- 172.21.15.62:0/637520911 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f660c0cdbb0 con 0x7f6610101060 2024-05-08T22:34:33.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.787+0000 7f661558f700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65fc0a11a0 0x7f65fc0a3660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6604005d90 tx=0x7f6604005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:33.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.790+0000 7f66027fc700 1 -- 172.21.15.62:0/637520911 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f660c09b420 con 0x7f6610101060 2024-05-08T22:34:33.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:33 smithi066 ceph-mon[46830]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:33.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:33 smithi066 ceph-mon[46830]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:34:33.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:33 smithi066 ceph-mon[46830]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T22:34:33.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:33 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:33.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:33 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:33.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:33 smithi066 ceph-mon[46830]: from='osd.11 ' entity='osd.11' 2024-05-08T22:34:33.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:33 smithi062 ceph-mon[42571]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:33.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:33 smithi062 ceph-mon[42571]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-05-08T22:34:33.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:33 smithi062 ceph-mon[42571]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T22:34:33.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:33 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:33.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:33 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:33.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:33 smithi062 ceph-mon[42571]: from='osd.11 ' entity='osd.11' 2024-05-08T22:34:33.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.992+0000 7f66177f3700 1 -- 172.21.15.62:0/637520911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f661010b760 con 0x7f6610101060 2024-05-08T22:34:33.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.992+0000 7f66027fc700 1 -- 172.21.15.62:0/637520911 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) v1 ==== 74+0+133 (secure 0 0 0) 0x7f660c09afb0 con 0x7f6610101060 2024-05-08T22:34:33.992 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:33.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 -- 172.21.15.62:0/637520911 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65fc0a11a0 msgr2=0x7f65fc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:33.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65fc0a11a0 0x7f65fc0a3660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6604005d90 tx=0x7f6604005d20 comp rx=0 tx=0).stop 2024-05-08T22:34:33.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 -- 172.21.15.62:0/637520911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610101060 msgr2=0x7f661019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:33.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610101060 0x7f661019bc50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f660c005950 tx=0x7f660c009580 comp rx=0 tx=0).stop 2024-05-08T22:34:33.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 -- 172.21.15.62:0/637520911 shutdown_connections 2024-05-08T22:34:33.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6610101a20 0x7f661019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65fc0a11a0 0x7f65fc0a3660 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66100fff00 0x7f6610069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 --2- 172.21.15.62:0/637520911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6610101060 0x7f661019bc50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:33.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 -- 172.21.15.62:0/637520911 >> 172.21.15.62:0/637520911 conn(0x7f66100fb4a0 msgr2=0x7f6610104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:33.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 -- 172.21.15.62:0/637520911 shutdown_connections 2024-05-08T22:34:33.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:33.994+0000 7f66177f3700 1 -- 172.21.15.62:0/637520911 wait complete. 2024-05-08T22:34:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1715207656,"num_in_osds":12,"osd_in_since":1715207656,"num_remapped_pgs":0} 2024-05-08T22:34:34.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:34 smithi066 ceph-mon[46830]: purged_snaps scrub starts 2024-05-08T22:34:34.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:34 smithi066 ceph-mon[46830]: purged_snaps scrub ok 2024-05-08T22:34:34.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:34 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:34.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:34 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/637520911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:34:34.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:34 smithi066 ceph-mon[46830]: osd.11 [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078] boot 2024-05-08T22:34:34.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:34 smithi066 ceph-mon[46830]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T22:34:34.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:34 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:34.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:34 smithi079 ceph-mon[46738]: purged_snaps scrub starts 2024-05-08T22:34:34.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:34 smithi079 ceph-mon[46738]: purged_snaps scrub ok 2024-05-08T22:34:34.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:34.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/637520911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:34:34.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:34 smithi079 ceph-mon[46738]: osd.11 [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078] boot 2024-05-08T22:34:34.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:34 smithi079 ceph-mon[46738]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T22:34:34.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:34 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:34.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:34 smithi062 ceph-mon[42571]: purged_snaps scrub starts 2024-05-08T22:34:34.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:34 smithi062 ceph-mon[42571]: purged_snaps scrub ok 2024-05-08T22:34:34.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:34 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:34.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:34 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/637520911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:34:34.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:34 smithi062 ceph-mon[42571]: osd.11 [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078] boot 2024-05-08T22:34:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:34 smithi062 ceph-mon[42571]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T22:34:34.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:34 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T22:34:35.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T22:34:35.711 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:34:35.818 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:35 smithi062 ceph-mon[42571]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:35.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:35 smithi066 ceph-mon[46830]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:35.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:35 smithi079 ceph-mon[46738]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-08T22:34:36.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:36 smithi062 ceph-mon[42571]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T22:34:36.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.706+0000 7f318e868700 1 -- 172.21.15.62:0/1044105402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3188104430 msgr2=0x7f3188106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.706+0000 7f318e868700 1 --2- 172.21.15.62:0/1044105402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3188104430 0x7f3188106820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3170009990 tx=0x7f3170009ca0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.706+0000 7f318e868700 1 -- 172.21.15.62:0/1044105402 shutdown_connections 2024-05-08T22:34:36.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.706+0000 7f318e868700 1 --2- 172.21.15.62:0/1044105402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3188106d60 0x7f31881091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.706+0000 7f318e868700 1 --2- 172.21.15.62:0/1044105402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3188101b00 0x7f3188103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.706+0000 7f318e868700 1 --2- 172.21.15.62:0/1044105402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3188104430 0x7f3188106820 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.706+0000 7f318e868700 1 -- 172.21.15.62:0/1044105402 >> 172.21.15.62:0/1044105402 conn(0x7f31880fb530 msgr2=0x7f31880fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:36.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.706+0000 7f318e868700 1 -- 172.21.15.62:0/1044105402 shutdown_connections 2024-05-08T22:34:36.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.706+0000 7f318e868700 1 -- 172.21.15.62:0/1044105402 wait complete. 2024-05-08T22:34:36.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f318e868700 1 Processor -- start 2024-05-08T22:34:36.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f318e868700 1 -- start start 2024-05-08T22:34:36.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f318e868700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3188101b00 0x7f3188101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f318e868700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3188104430 0x7f31880ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f318e868700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3188106d60 0x7f31880ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f318e868700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3188100790 con 0x7f3188106d60 2024-05-08T22:34:36.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f318e868700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3188100900 con 0x7f3188101b00 2024-05-08T22:34:36.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f318e868700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881a1050 con 0x7f3188104430 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f318ce05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3188106d60 0x7f31880ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f31877fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3188104430 0x7f31880ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f31877fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3188104430 0x7f31880ffab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.62:38074/0 (socket says 172.21.15.62:38074) 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f318ce05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3188106d60 0x7f31880ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34738/0 (socket says 172.21.15.62:34738) 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f31877fe700 1 -- 172.21.15.62:0/3930909812 learned_addr learned my addr 172.21.15.62:0/3930909812 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.707+0000 7f3187fff700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3188101b00 0x7f3188101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.708+0000 7f318ce05700 1 -- 172.21.15.62:0/3930909812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3188104430 msgr2=0x7f31880ffab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.708+0000 7f318ce05700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3188104430 0x7f31880ffab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.708+0000 7f318ce05700 1 -- 172.21.15.62:0/3930909812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3188101b00 msgr2=0x7f3188101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.708+0000 7f318ce05700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3188101b00 0x7f3188101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.708+0000 7f318ce05700 1 -- 172.21.15.62:0/3930909812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31700095f0 con 0x7f3188106d60 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.708+0000 7f318ce05700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3188106d60 0x7f31880ffff0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3180007ee0 tx=0x7f318000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.708+0000 7f31857fa700 1 -- 172.21.15.62:0/3930909812 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180004db0 con 0x7f3188106d60 2024-05-08T22:34:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.708+0000 7f31857fa700 1 -- 172.21.15.62:0/3930909812 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f318000f040 con 0x7f3188106d60 2024-05-08T22:34:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.708+0000 7f318e868700 1 -- 172.21.15.62:0/3930909812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31881a12e0 con 0x7f3188106d60 2024-05-08T22:34:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.708+0000 7f31857fa700 1 -- 172.21.15.62:0/3930909812 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180005900 con 0x7f3188106d60 2024-05-08T22:34:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.708+0000 7f318e868700 1 -- 172.21.15.62:0/3930909812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31881a1790 con 0x7f3188106d60 2024-05-08T22:34:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.710+0000 7f318e868700 1 -- 172.21.15.62:0/3930909812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31880fd0b0 con 0x7f3188106d60 2024-05-08T22:34:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.710+0000 7f31857fa700 1 -- 172.21.15.62:0/3930909812 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f318000db20 con 0x7f3188106d60 2024-05-08T22:34:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.711+0000 7f31857fa700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f31740a1120 0x7f31740a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.711+0000 7f3187fff700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f31740a1120 0x7f31740a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.711+0000 7f31857fa700 1 -- 172.21.15.62:0/3930909812 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f31800cbce0 con 0x7f3188106d60 2024-05-08T22:34:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.711+0000 7f3187fff700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f31740a1120 0x7f31740a35e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3178005950 tx=0x7f317800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:36.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.715+0000 7f31857fa700 1 -- 172.21.15.62:0/3930909812 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3180095910 con 0x7f3188106d60 2024-05-08T22:34:36.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:36 smithi066 ceph-mon[46830]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T22:34:36.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:36 smithi079 ceph-mon[46738]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T22:34:36.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.918+0000 7f318e868700 1 -- 172.21.15.62:0/3930909812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f31880fca90 con 0x7f3188106d60 2024-05-08T22:34:36.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.918+0000 7f31857fa700 1 -- 172.21.15.62:0/3930909812 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f3180098f30 con 0x7f3188106d60 2024-05-08T22:34:36.919 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 -- 172.21.15.62:0/3930909812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f31740a1120 msgr2=0x7f31740a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f31740a1120 0x7f31740a35e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3178005950 tx=0x7f317800b410 comp rx=0 tx=0).stop 2024-05-08T22:34:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 -- 172.21.15.62:0/3930909812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3188106d60 msgr2=0x7f31880ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3188106d60 0x7f31880ffff0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3180007ee0 tx=0x7f318000d3b0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 -- 172.21.15.62:0/3930909812 shutdown_connections 2024-05-08T22:34:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3188106d60 0x7f31880ffff0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f31740a1120 0x7f31740a35e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3188101b00 0x7f3188101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 --2- 172.21.15.62:0/3930909812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3188104430 0x7f31880ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 -- 172.21.15.62:0/3930909812 >> 172.21.15.62:0/3930909812 conn(0x7f31880fb530 msgr2=0x7f3188102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 -- 172.21.15.62:0/3930909812 shutdown_connections 2024-05-08T22:34:36.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:36.921+0000 7f318e868700 1 -- 172.21.15.62:0/3930909812 wait complete. 2024-05-08T22:34:37.282 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1715207674,"num_in_osds":12,"osd_in_since":1715207656,"num_remapped_pgs":0} 2024-05-08T22:34:37.283 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T22:34:37.509 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:34:37.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:37 smithi066 ceph-mon[46830]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:37.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:37 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3930909812' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:34:37.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:37 smithi062 ceph-mon[42571]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:37.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:37 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3930909812' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:34:37.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:37 smithi079 ceph-mon[46738]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:37.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:37 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3930909812' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.714+0000 7f229b3aa700 1 -- 172.21.15.62:0/688041174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2294107020 msgr2=0x7f2294107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.714+0000 7f229b3aa700 1 --2- 172.21.15.62:0/688041174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2294107020 0x7f2294107400 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2290009990 tx=0x7f2290009ca0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.714+0000 7f229b3aa700 1 -- 172.21.15.62:0/688041174 shutdown_connections 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.714+0000 7f229b3aa700 1 --2- 172.21.15.62:0/688041174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22940692f0 0x7f229410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.714+0000 7f229b3aa700 1 --2- 172.21.15.62:0/688041174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2294107020 0x7f2294107400 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.714+0000 7f229b3aa700 1 --2- 172.21.15.62:0/688041174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2294068950 0x7f2294068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.714+0000 7f229b3aa700 1 -- 172.21.15.62:0/688041174 >> 172.21.15.62:0/688041174 conn(0x7f2294075f80 msgr2=0x7f2294076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.714+0000 7f229b3aa700 1 -- 172.21.15.62:0/688041174 shutdown_connections 2024-05-08T22:34:38.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.714+0000 7f229b3aa700 1 -- 172.21.15.62:0/688041174 wait complete. 2024-05-08T22:34:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f229b3aa700 1 Processor -- start 2024-05-08T22:34:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f229b3aa700 1 -- start start 2024-05-08T22:34:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f229b3aa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2294068950 0x7f22940fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f229b3aa700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22940692f0 0x7f22941004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f229b3aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2294107020 0x7f2294100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f229b3aa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22941a57d0 con 0x7f2294107020 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f229b3aa700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22941a5940 con 0x7f22940692f0 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f229b3aa700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22941a5b00 con 0x7f2294068950 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f2299947700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2294107020 0x7f2294100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f2299947700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2294107020 0x7f2294100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34750/0 (socket says 172.21.15.62:34750) 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f2299947700 1 -- 172.21.15.62:0/827306071 learned_addr learned my addr 172.21.15.62:0/827306071 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f2299146700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2294068950 0x7f22940fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f2298945700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22940692f0 0x7f22941004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f2299947700 1 -- 172.21.15.62:0/827306071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2294068950 msgr2=0x7f22940fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f2299947700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2294068950 0x7f22940fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f2299947700 1 -- 172.21.15.62:0/827306071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22940692f0 msgr2=0x7f22941004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f2299947700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22940692f0 0x7f22941004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f2299947700 1 -- 172.21.15.62:0/827306071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900095f0 con 0x7f2294107020 2024-05-08T22:34:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.715+0000 7f2298945700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22940692f0 0x7f22941004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.716+0000 7f2299947700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2294107020 0x7f2294100a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f228c007ee0 tx=0x7f228c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.716+0000 7f228a7fc700 1 -- 172.21.15.62:0/827306071 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c00ddd0 con 0x7f2294107020 2024-05-08T22:34:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.716+0000 7f229b3aa700 1 -- 172.21.15.62:0/827306071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22941a5de0 con 0x7f2294107020 2024-05-08T22:34:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.716+0000 7f228a7fc700 1 -- 172.21.15.62:0/827306071 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f228c00f040 con 0x7f2294107020 2024-05-08T22:34:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.716+0000 7f228a7fc700 1 -- 172.21.15.62:0/827306071 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c0048e0 con 0x7f2294107020 2024-05-08T22:34:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.716+0000 7f229b3aa700 1 -- 172.21.15.62:0/827306071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22941a62b0 con 0x7f2294107020 2024-05-08T22:34:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.717+0000 7f229b3aa700 1 -- 172.21.15.62:0/827306071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2294109540 con 0x7f2294107020 2024-05-08T22:34:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.719+0000 7f228a7fc700 1 -- 172.21.15.62:0/827306071 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f228c003a00 con 0x7f2294107020 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.719+0000 7f228a7fc700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f22800a1120 0x7f22800a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.719+0000 7f2299146700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f22800a1120 0x7f22800a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.719+0000 7f228a7fc700 1 -- 172.21.15.62:0/827306071 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f228c0cc500 con 0x7f2294107020 2024-05-08T22:34:38.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.720+0000 7f2299146700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f22800a1120 0x7f22800a35e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22900098e0 tx=0x7f2290005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:38.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.722+0000 7f228a7fc700 1 -- 172.21.15.62:0/827306071 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f228c096130 con 0x7f2294107020 2024-05-08T22:34:38.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.928+0000 7f229b3aa700 1 -- 172.21.15.62:0/827306071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2294101480 con 0x7f2294107020 2024-05-08T22:34:38.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.929+0000 7f228a7fc700 1 -- 172.21.15.62:0/827306071 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f228c099750 con 0x7f2294107020 2024-05-08T22:34:38.929 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:38.929 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":75,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:34:35.551157+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 -- 172.21.15.62:0/827306071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f22800a1120 msgr2=0x7f22800a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f22800a1120 0x7f22800a35e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22900098e0 tx=0x7f2290005960 comp rx=0 tx=0).stop 2024-05-08T22:34:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 -- 172.21.15.62:0/827306071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2294107020 msgr2=0x7f2294100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2294107020 0x7f2294100a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f228c007ee0 tx=0x7f228c00d3b0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 -- 172.21.15.62:0/827306071 shutdown_connections 2024-05-08T22:34:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2294107020 0x7f2294100a20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2294068950 0x7f22940fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f22800a1120 0x7f22800a35e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 --2- 172.21.15.62:0/827306071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22940692f0 0x7f22941004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:38.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 -- 172.21.15.62:0/827306071 >> 172.21.15.62:0/827306071 conn(0x7f2294075f80 msgr2=0x7f22940fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:38.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 -- 172.21.15.62:0/827306071 shutdown_connections 2024-05-08T22:34:38.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:38.931+0000 7f229b3aa700 1 -- 172.21.15.62:0/827306071 wait complete. 2024-05-08T22:34:39.424 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:31:54.466249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:34:39.424 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-08T22:34:39.655 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:34:39.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:39 smithi062 ceph-mon[42571]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:39.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:39 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/827306071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:34:39.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:39 smithi066 ceph-mon[46830]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:39.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:39 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/827306071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:34:39.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:39 smithi079 ceph-mon[46738]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:39.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:39 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/827306071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:34:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.684+0000 7f9fcba70700 1 -- 172.21.15.62:0/2149675453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc40692f0 msgr2=0x7f9fc410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.684+0000 7f9fcba70700 1 --2- 172.21.15.62:0/2149675453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc40692f0 0x7f9fc410de60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00b0a0 tx=0x7f9fbc00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.684+0000 7f9fcba70700 1 -- 172.21.15.62:0/2149675453 shutdown_connections 2024-05-08T22:34:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.684+0000 7f9fcba70700 1 --2- 172.21.15.62:0/2149675453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc40692f0 0x7f9fc410de60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.684+0000 7f9fcba70700 1 --2- 172.21.15.62:0/2149675453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fc4107020 0x7f9fc4107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.684+0000 7f9fcba70700 1 --2- 172.21.15.62:0/2149675453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc4068950 0x7f9fc4068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.684+0000 7f9fcba70700 1 -- 172.21.15.62:0/2149675453 >> 172.21.15.62:0/2149675453 conn(0x7f9fc4075f80 msgr2=0x7f9fc4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.684+0000 7f9fcba70700 1 -- 172.21.15.62:0/2149675453 shutdown_connections 2024-05-08T22:34:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.685+0000 7f9fcba70700 1 -- 172.21.15.62:0/2149675453 wait complete. 2024-05-08T22:34:40.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.685+0000 7f9fcba70700 1 Processor -- start 2024-05-08T22:34:40.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.685+0000 7f9fcba70700 1 -- start start 2024-05-08T22:34:40.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fcba70700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4068950 0x7f9fc40fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fcba70700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fc40692f0 0x7f9fc4100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fcba70700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc4107020 0x7f9fc4100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fcba70700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc41a5820 con 0x7f9fc4068950 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fcba70700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc41a5990 con 0x7f9fc4107020 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fcba70700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc41a5b50 con 0x7f9fc40692f0 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fc980c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4068950 0x7f9fc40fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fc980c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4068950 0x7f9fc40fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34766/0 (socket says 172.21.15.62:34766) 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fc980c700 1 -- 172.21.15.62:0/1503978366 learned_addr learned my addr 172.21.15.62:0/1503978366 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fca00d700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc4107020 0x7f9fc4100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fc900b700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fc40692f0 0x7f9fc4100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fc980c700 1 -- 172.21.15.62:0/1503978366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fc40692f0 msgr2=0x7f9fc4100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fc980c700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fc40692f0 0x7f9fc4100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fc980c700 1 -- 172.21.15.62:0/1503978366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc4107020 msgr2=0x7f9fc4100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fc980c700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc4107020 0x7f9fc4100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fc980c700 1 -- 172.21.15.62:0/1503978366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc009d00 con 0x7f9fc4068950 2024-05-08T22:34:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fc900b700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fc40692f0 0x7f9fc4100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fc980c700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4068950 0x7f9fc40fffc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000c9b0 tx=0x7f9fc000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.686+0000 7f9fb6ffd700 1 -- 172.21.15.62:0/1503978366 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0007b00 con 0x7f9fc4068950 2024-05-08T22:34:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.687+0000 7f9fcba70700 1 -- 172.21.15.62:0/1503978366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc41a5e30 con 0x7f9fc4068950 2024-05-08T22:34:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.687+0000 7f9fb6ffd700 1 -- 172.21.15.62:0/1503978366 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fc0007c60 con 0x7f9fc4068950 2024-05-08T22:34:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.687+0000 7f9fb6ffd700 1 -- 172.21.15.62:0/1503978366 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0018660 con 0x7f9fc4068950 2024-05-08T22:34:40.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.687+0000 7f9fcba70700 1 -- 172.21.15.62:0/1503978366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc41a62a0 con 0x7f9fc4068950 2024-05-08T22:34:40.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.688+0000 7f9fcba70700 1 -- 172.21.15.62:0/1503978366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc4109540 con 0x7f9fc4068950 2024-05-08T22:34:40.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.689+0000 7f9fb6ffd700 1 -- 172.21.15.62:0/1503978366 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9fc000f450 con 0x7f9fc4068950 2024-05-08T22:34:40.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.689+0000 7f9fb6ffd700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9fb00a11a0 0x7f9fb00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:40.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.689+0000 7f9fc900b700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9fb00a11a0 0x7f9fb00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:40.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.689+0000 7f9fb6ffd700 1 -- 172.21.15.62:0/1503978366 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9fc00cc2f0 con 0x7f9fc4068950 2024-05-08T22:34:40.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.690+0000 7f9fc900b700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9fb00a11a0 0x7f9fb00a3660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4101690 tx=0x7f9fb8005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:40.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.694+0000 7f9fb6ffd700 1 -- 172.21.15.62:0/1503978366 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9fc0095f20 con 0x7f9fc4068950 2024-05-08T22:34:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.892+0000 7f9fcba70700 1 -- 172.21.15.62:0/1503978366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9fc41026d0 con 0x7f9fc4068950 2024-05-08T22:34:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.893+0000 7f9fb6ffd700 1 -- 172.21.15.62:0/1503978366 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9fc41026d0 con 0x7f9fc4068950 2024-05-08T22:34:40.893 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-05-08T22:34:40.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 -- 172.21.15.62:0/1503978366 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9fb00a11a0 msgr2=0x7f9fb00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:40.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9fb00a11a0 0x7f9fb00a3660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4101690 tx=0x7f9fb8005f50 comp rx=0 tx=0).stop 2024-05-08T22:34:40.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 -- 172.21.15.62:0/1503978366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4068950 msgr2=0x7f9fc40fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:40.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4068950 0x7f9fc40fffc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000c9b0 tx=0x7f9fc000ccc0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 -- 172.21.15.62:0/1503978366 shutdown_connections 2024-05-08T22:34:40.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9fb00a11a0 0x7f9fb00a3660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fc4107020 0x7f9fc4100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4068950 0x7f9fc40fffc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 --2- 172.21.15.62:0/1503978366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9fc40692f0 0x7f9fc4100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 -- 172.21.15.62:0/1503978366 >> 172.21.15.62:0/1503978366 conn(0x7f9fc4075f80 msgr2=0x7f9fc40fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 -- 172.21.15.62:0/1503978366 shutdown_connections 2024-05-08T22:34:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:40.895+0000 7f9fcba70700 1 -- 172.21.15.62:0/1503978366 wait complete. 2024-05-08T22:34:41.220 INFO:tasks.cephadm:Adding mds.a on smithi062 2024-05-08T22:34:41.221 INFO:tasks.cephadm:Adding mds.d on smithi062 2024-05-08T22:34:41.221 INFO:tasks.cephadm:Adding mds.g on smithi062 2024-05-08T22:34:41.221 INFO:tasks.cephadm:Adding mds.j on smithi062 2024-05-08T22:34:41.221 INFO:tasks.cephadm:Adding mds.b on smithi066 2024-05-08T22:34:41.221 INFO:tasks.cephadm:Adding mds.e on smithi066 2024-05-08T22:34:41.221 INFO:tasks.cephadm:Adding mds.h on smithi066 2024-05-08T22:34:41.221 INFO:tasks.cephadm:Adding mds.k on smithi066 2024-05-08T22:34:41.221 INFO:tasks.cephadm:Adding mds.c on smithi079 2024-05-08T22:34:41.221 INFO:tasks.cephadm:Adding mds.f on smithi079 2024-05-08T22:34:41.221 INFO:tasks.cephadm:Adding mds.i on smithi079 2024-05-08T22:34:41.221 INFO:tasks.cephadm:Adding mds.l on smithi079 2024-05-08T22:34:41.222 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l' 2024-05-08T22:34:41.456 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:34:41.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:41 smithi066 ceph-mon[46830]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:41.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:41 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1503978366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:34:41.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:41 smithi079 ceph-mon[46738]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:41.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:41 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1503978366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:34:41.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:41 smithi062 ceph-mon[42571]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:41.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:41 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1503978366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:34:42.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.442+0000 7f69fd40c700 1 -- 172.21.15.79:0/999424378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69f8102b80 msgr2=0x7f69f8109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:42.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.442+0000 7f69fd40c700 1 --2- 172.21.15.79:0/999424378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69f8102b80 0x7f69f8109430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f69f000b0a0 tx=0x7f69f000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.442+0000 7f69fd40c700 1 -- 172.21.15.79:0/999424378 shutdown_connections 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.442+0000 7f69fd40c700 1 --2- 172.21.15.79:0/999424378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69f8102b80 0x7f69f8109430 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.442+0000 7f69fd40c700 1 --2- 172.21.15.79:0/999424378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69f8100fe0 0x7f69f8101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.442+0000 7f69fd40c700 1 --2- 172.21.15.79:0/999424378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69f81021e0 0x7f69f8102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.442+0000 7f69fd40c700 1 -- 172.21.15.79:0/999424378 >> 172.21.15.79:0/999424378 conn(0x7f69f80fc580 msgr2=0x7f69f80fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.443+0000 7f69fd40c700 1 -- 172.21.15.79:0/999424378 shutdown_connections 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.443+0000 7f69fd40c700 1 -- 172.21.15.79:0/999424378 wait complete. 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.443+0000 7f69fd40c700 1 Processor -- start 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.443+0000 7f69fd40c700 1 -- start start 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.443+0000 7f69fd40c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69f8100fe0 0x7f69f8074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.443+0000 7f69fd40c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69f81021e0 0x7f69f8073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.443+0000 7f69fd40c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69f8102b80 0x7f69f80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.443+0000 7f69fd40c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8074070 con 0x7f69f81021e0 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.443+0000 7f69fd40c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f80741e0 con 0x7f69f8100fe0 2024-05-08T22:34:42.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.443+0000 7f69fd40c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f81a1120 con 0x7f69f8102b80 2024-05-08T22:34:42.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69f8102b80 0x7f69f80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:42.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69f8102b80 0x7f69f80738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:59442/0 (socket says 172.21.15.79:59442) 2024-05-08T22:34:42.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f77fe700 1 -- 172.21.15.79:0/539806298 learned_addr learned my addr 172.21.15.79:0/539806298 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-05-08T22:34:42.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f6ffd700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69f8100fe0 0x7f69f8074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:42.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f77fe700 1 -- 172.21.15.79:0/539806298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69f8100fe0 msgr2=0x7f69f8074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:42.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f77fe700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69f8100fe0 0x7f69f8074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f77fe700 1 -- 172.21.15.79:0/539806298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69f81021e0 msgr2=0x7f69f8073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:42.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f67fc700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69f81021e0 0x7f69f8073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:42.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f77fe700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69f81021e0 0x7f69f8073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f77fe700 1 -- 172.21.15.79:0/539806298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f0009d00 con 0x7f69f8102b80 2024-05-08T22:34:42.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f67fc700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69f81021e0 0x7f69f8073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:42.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69f77fe700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69f8102b80 0x7f69f80738d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f69f000b830 tx=0x7f69f0004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:42.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69dffff700 1 -- 172.21.15.79:0/539806298 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f000e040 con 0x7f69f8102b80 2024-05-08T22:34:42.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69dffff700 1 -- 172.21.15.79:0/539806298 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69f001db50 con 0x7f69f8102b80 2024-05-08T22:34:42.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69fd40c700 1 -- 172.21.15.79:0/539806298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f81a1350 con 0x7f69f8102b80 2024-05-08T22:34:42.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69dffff700 1 -- 172.21.15.79:0/539806298 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f001bbf0 con 0x7f69f8102b80 2024-05-08T22:34:42.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.444+0000 7f69fd40c700 1 -- 172.21.15.79:0/539806298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f81a17c0 con 0x7f69f8102b80 2024-05-08T22:34:42.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.445+0000 7f69fd40c700 1 -- 172.21.15.79:0/539806298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f81083c0 con 0x7f69f8102b80 2024-05-08T22:34:42.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.446+0000 7f69dffff700 1 -- 172.21.15.79:0/539806298 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f69f001bd50 con 0x7f69f8102b80 2024-05-08T22:34:42.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.447+0000 7f69dffff700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69e00a1120 0x7f69e00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:42.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.447+0000 7f69dffff700 1 -- 172.21.15.79:0/539806298 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f69f00ce710 con 0x7f69f8102b80 2024-05-08T22:34:42.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.447+0000 7f69f6ffd700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69e00a1120 0x7f69e00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:42.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.448+0000 7f69f6ffd700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69e00a1120 0x7f69e00a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69f8102040 tx=0x7f69e8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:42.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.451+0000 7f69dffff700 1 -- 172.21.15.79:0/539806298 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f69f0098330 con 0x7f69f8102b80 2024-05-08T22:34:42.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.656+0000 7f69fd40c700 1 -- 172.21.15.79:0/539806298 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f69f802cf10 con 0x7f69e00a1120 2024-05-08T22:34:42.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.664+0000 7f69dffff700 1 -- 172.21.15.79:0/539806298 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f69f802cf10 con 0x7f69e00a1120 2024-05-08T22:34:42.664 INFO:teuthology.orchestra.run.smithi079.stdout:Scheduled mds.all update... 2024-05-08T22:34:42.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.666+0000 7f69fd40c700 1 -- 172.21.15.79:0/539806298 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69e00a1120 msgr2=0x7f69e00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:42.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.666+0000 7f69fd40c700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69e00a1120 0x7f69e00a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69f8102040 tx=0x7f69e8009190 comp rx=0 tx=0).stop 2024-05-08T22:34:42.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.666+0000 7f69fd40c700 1 -- 172.21.15.79:0/539806298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69f8102b80 msgr2=0x7f69f80738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:42.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.666+0000 7f69fd40c700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69f8102b80 0x7f69f80738d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f69f000b830 tx=0x7f69f0004c80 comp rx=0 tx=0).stop 2024-05-08T22:34:42.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.666+0000 7f69fd40c700 1 -- 172.21.15.79:0/539806298 shutdown_connections 2024-05-08T22:34:42.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.666+0000 7f69fd40c700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69f8102b80 0x7f69f80738d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.666+0000 7f69fd40c700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69e00a1120 0x7f69e00a35e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.666+0000 7f69fd40c700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69f8100fe0 0x7f69f8074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.666+0000 7f69fd40c700 1 --2- 172.21.15.79:0/539806298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69f81021e0 0x7f69f8073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:42.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.666+0000 7f69fd40c700 1 -- 172.21.15.79:0/539806298 >> 172.21.15.79:0/539806298 conn(0x7f69f80fc580 msgr2=0x7f69f8105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:42.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.667+0000 7f69fd40c700 1 -- 172.21.15.79:0/539806298 shutdown_connections 2024-05-08T22:34:42.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-05-08T22:34:42.667+0000 7f69fd40c700 1 -- 172.21.15.79:0/539806298 wait complete. 2024-05-08T22:34:43.067 DEBUG:teuthology.orchestra.run.smithi062:mds.a> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.a.service 2024-05-08T22:34:43.070 DEBUG:teuthology.orchestra.run.smithi062:mds.d> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.d.service 2024-05-08T22:34:43.102 INFO:journalctl@ceph.mds.a.smithi062.stdout:-- Logs begin at Wed 2024-05-08 22:12:07 UTC. -- 2024-05-08T22:34:43.103 DEBUG:teuthology.orchestra.run.smithi062:mds.g> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.g.service 2024-05-08T22:34:43.130 INFO:journalctl@ceph.mds.d.smithi062.stdout:-- Logs begin at Wed 2024-05-08 22:12:07 UTC. -- 2024-05-08T22:34:43.131 DEBUG:teuthology.orchestra.run.smithi062:mds.j> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.j.service 2024-05-08T22:34:43.160 INFO:journalctl@ceph.mds.g.smithi062.stdout:-- Logs begin at Wed 2024-05-08 22:12:07 UTC. -- 2024-05-08T22:34:43.161 DEBUG:teuthology.orchestra.run.smithi066:mds.b> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.b.service 2024-05-08T22:34:43.163 DEBUG:teuthology.orchestra.run.smithi066:mds.e> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.e.service 2024-05-08T22:34:43.188 INFO:journalctl@ceph.mds.j.smithi062.stdout:-- Logs begin at Wed 2024-05-08 22:12:07 UTC. -- 2024-05-08T22:34:43.205 INFO:journalctl@ceph.mds.b.smithi066.stdout:-- Logs begin at Wed 2024-05-08 22:13:17 UTC. -- 2024-05-08T22:34:43.206 DEBUG:teuthology.orchestra.run.smithi066:mds.h> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.h.service 2024-05-08T22:34:43.237 INFO:journalctl@ceph.mds.e.smithi066.stdout:-- Logs begin at Wed 2024-05-08 22:13:17 UTC. -- 2024-05-08T22:34:43.238 DEBUG:teuthology.orchestra.run.smithi066:mds.k> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.k.service 2024-05-08T22:34:43.267 INFO:journalctl@ceph.mds.h.smithi066.stdout:-- Logs begin at Wed 2024-05-08 22:13:17 UTC. -- 2024-05-08T22:34:43.268 DEBUG:teuthology.orchestra.run.smithi079:mds.c> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.c.service 2024-05-08T22:34:43.270 DEBUG:teuthology.orchestra.run.smithi079:mds.f> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.f.service 2024-05-08T22:34:43.296 INFO:journalctl@ceph.mds.k.smithi066.stdout:-- Logs begin at Wed 2024-05-08 22:13:17 UTC. -- 2024-05-08T22:34:43.303 INFO:journalctl@ceph.mds.c.smithi079.stdout:-- Logs begin at Wed 2024-05-08 22:12:11 UTC. -- 2024-05-08T22:34:43.304 DEBUG:teuthology.orchestra.run.smithi079:mds.i> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.i.service 2024-05-08T22:34:43.331 INFO:journalctl@ceph.mds.f.smithi079.stdout:-- Logs begin at Wed 2024-05-08 22:12:11 UTC. -- 2024-05-08T22:34:43.333 DEBUG:teuthology.orchestra.run.smithi079:mds.l> sudo journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.l.service 2024-05-08T22:34:43.360 INFO:journalctl@ceph.mds.i.smithi079.stdout:-- Logs begin at Wed 2024-05-08 22:12:11 UTC. -- 2024-05-08T22:34:43.361 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-08T22:34:43.362 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:34:43.388 INFO:journalctl@ceph.mds.l.smithi079.stdout:-- Logs begin at Wed 2024-05-08 22:12:11 UTC. -- 2024-05-08T22:34:43.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.510+0000 7f8f896bc700 1 -- 172.21.15.62:0/3498625373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f84106b50 msgr2=0x7f8f84108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.510+0000 7f8f896bc700 1 --2- 172.21.15.62:0/3498625373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f84106b50 0x7f8f84108fc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00b0a0 tx=0x7f8f7c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.510+0000 7f8f896bc700 1 -- 172.21.15.62:0/3498625373 shutdown_connections 2024-05-08T22:34:43.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.510+0000 7f8f896bc700 1 --2- 172.21.15.62:0/3498625373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f84106b50 0x7f8f84108fc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.510+0000 7f8f896bc700 1 --2- 172.21.15.62:0/3498625373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f841018f0 0x7f8f84103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.510+0000 7f8f896bc700 1 --2- 172.21.15.62:0/3498625373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f84104220 0x7f8f84106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.510+0000 7f8f896bc700 1 -- 172.21.15.62:0/3498625373 >> 172.21.15.62:0/3498625373 conn(0x7f8f840fb2e0 msgr2=0x7f8f840fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:43.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.510+0000 7f8f896bc700 1 -- 172.21.15.62:0/3498625373 shutdown_connections 2024-05-08T22:34:43.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.510+0000 7f8f896bc700 1 -- 172.21.15.62:0/3498625373 wait complete. 2024-05-08T22:34:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.510+0000 7f8f896bc700 1 Processor -- start 2024-05-08T22:34:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f896bc700 1 -- start start 2024-05-08T22:34:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f896bc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f841018f0 0x7f8f84101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f896bc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f84104220 0x7f8f840ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f896bc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f84106b50 0x7f8f840ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f896bc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f84100580 con 0x7f8f841018f0 2024-05-08T22:34:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f896bc700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f841006f0 con 0x7f8f84104220 2024-05-08T22:34:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f896bc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f841a0f10 con 0x7f8f84106b50 2024-05-08T22:34:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f82ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f841018f0 0x7f8f84101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f82ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f841018f0 0x7f8f84101210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34790/0 (socket says 172.21.15.62:34790) 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f82ffd700 1 -- 172.21.15.62:0/296609234 learned_addr learned my addr 172.21.15.62:0/296609234 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f837fe700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f84106b50 0x7f8f840ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f827fc700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f84104220 0x7f8f840ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f82ffd700 1 -- 172.21.15.62:0/296609234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f84106b50 msgr2=0x7f8f840ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f82ffd700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f84106b50 0x7f8f840ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f82ffd700 1 -- 172.21.15.62:0/296609234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f84104220 msgr2=0x7f8f840ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f82ffd700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f84104220 0x7f8f840ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f82ffd700 1 -- 172.21.15.62:0/296609234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f78009520 con 0x7f8f841018f0 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f827fc700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f84104220 0x7f8f840ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f82ffd700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f841018f0 0x7f8f84101210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400d950 tx=0x7f8f7400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.511+0000 7f8f6bfff700 1 -- 172.21.15.62:0/296609234 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f740098e0 con 0x7f8f841018f0 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.512+0000 7f8f6bfff700 1 -- 172.21.15.62:0/296609234 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f74010460 con 0x7f8f841018f0 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.512+0000 7f8f896bc700 1 -- 172.21.15.62:0/296609234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7c009d00 con 0x7f8f841018f0 2024-05-08T22:34:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.512+0000 7f8f6bfff700 1 -- 172.21.15.62:0/296609234 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400f620 con 0x7f8f841018f0 2024-05-08T22:34:43.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.512+0000 7f8f896bc700 1 -- 172.21.15.62:0/296609234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f841a1520 con 0x7f8f841018f0 2024-05-08T22:34:43.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.512+0000 7f8f896bc700 1 -- 172.21.15.62:0/296609234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f840fce60 con 0x7f8f841018f0 2024-05-08T22:34:43.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.513+0000 7f8f6bfff700 1 -- 172.21.15.62:0/296609234 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8f740108d0 con 0x7f8f841018f0 2024-05-08T22:34:43.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.514+0000 7f8f6bfff700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f6c0a1120 0x7f8f6c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.514+0000 7f8f827fc700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f6c0a1120 0x7f8f6c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.514+0000 7f8f6bfff700 1 -- 172.21.15.62:0/296609234 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8f740cc2b0 con 0x7f8f841018f0 2024-05-08T22:34:43.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.515+0000 7f8f827fc700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f6c0a1120 0x7f8f6c0a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009d70 tx=0x7f8f78009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:43.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.518+0000 7f8f6bfff700 1 -- 172.21.15.62:0/296609234 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8f74095ee0 con 0x7f8f841018f0 2024-05-08T22:34:43.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.729+0000 7f8f896bc700 1 -- 172.21.15.62:0/296609234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f84068160 con 0x7f8f841018f0 2024-05-08T22:34:43.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.730+0000 7f8f6bfff700 1 -- 172.21.15.62:0/296609234 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f8f74099500 con 0x7f8f841018f0 2024-05-08T22:34:43.730 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:43.731 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":75,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:34:35.551157+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:34:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.732+0000 7f8f896bc700 1 -- 172.21.15.62:0/296609234 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f6c0a1120 msgr2=0x7f8f6c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.732+0000 7f8f896bc700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f6c0a1120 0x7f8f6c0a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009d70 tx=0x7f8f78009260 comp rx=0 tx=0).stop 2024-05-08T22:34:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.732+0000 7f8f896bc700 1 -- 172.21.15.62:0/296609234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f841018f0 msgr2=0x7f8f84101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.732+0000 7f8f896bc700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f841018f0 0x7f8f84101210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400d950 tx=0x7f8f7400dc60 comp rx=0 tx=0).stop 2024-05-08T22:34:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.732+0000 7f8f896bc700 1 -- 172.21.15.62:0/296609234 shutdown_connections 2024-05-08T22:34:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.732+0000 7f8f896bc700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f84106b50 0x7f8f840ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.732+0000 7f8f896bc700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f841018f0 0x7f8f84101210 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.732+0000 7f8f896bc700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8f6c0a1120 0x7f8f6c0a35e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.732+0000 7f8f896bc700 1 --2- 172.21.15.62:0/296609234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f84104220 0x7f8f840ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.733+0000 7f8f896bc700 1 -- 172.21.15.62:0/296609234 >> 172.21.15.62:0/296609234 conn(0x7f8f840fb2e0 msgr2=0x7f8f84102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:43.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.733+0000 7f8f896bc700 1 -- 172.21.15.62:0/296609234 shutdown_connections 2024-05-08T22:34:43.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.733+0000 7f8f896bc700 1 -- 172.21.15.62:0/296609234 wait complete. 2024-05-08T22:34:43.746 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:31:54.466249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:34:43.746 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:34:43.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.875+0000 7fb4f1964700 1 -- 172.21.15.62:0/827010969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4ec102a90 msgr2=0x7fb4ec109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.875+0000 7fb4f1964700 1 --2- 172.21.15.62:0/827010969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4ec102a90 0x7fb4ec109340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400b0a0 tx=0x7fb4e400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.875+0000 7fb4f1964700 1 -- 172.21.15.62:0/827010969 shutdown_connections 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.875+0000 7fb4f1964700 1 --2- 172.21.15.62:0/827010969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4ec102a90 0x7fb4ec109340 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.875+0000 7fb4f1964700 1 --2- 172.21.15.62:0/827010969 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4ec0ff120 0x7fb4ec0ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.875+0000 7fb4f1964700 1 --2- 172.21.15.62:0/827010969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ec1020f0 0x7fb4ec102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.875+0000 7fb4f1964700 1 -- 172.21.15.62:0/827010969 >> 172.21.15.62:0/827010969 conn(0x7fb4ec0faa50 msgr2=0x7fb4ec0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.875+0000 7fb4f1964700 1 -- 172.21.15.62:0/827010969 shutdown_connections 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.875+0000 7fb4f1964700 1 -- 172.21.15.62:0/827010969 wait complete. 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4f1964700 1 Processor -- start 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4f1964700 1 -- start start 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4f1964700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ec0ff120 0x7fb4ec101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4f1964700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4ec1020f0 0x7fb4ec1000a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4f1964700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4ec102a90 0x7fb4ec1005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4f1964700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec100d80 con 0x7fb4ec1020f0 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4f1964700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec100ef0 con 0x7fb4ec0ff120 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4f1964700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1a0f20 con 0x7fb4ec102a90 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4ea7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4ec1020f0 0x7fb4ec1000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4ea7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4ec1020f0 0x7fb4ec1000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34812/0 (socket says 172.21.15.62:34812) 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4ea7fc700 1 -- 172.21.15.62:0/2387136663 learned_addr learned my addr 172.21.15.62:0/2387136663 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4eb7fe700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4ec102a90 0x7fb4ec1005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4eaffd700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ec0ff120 0x7fb4ec101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4ea7fc700 1 -- 172.21.15.62:0/2387136663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4ec102a90 msgr2=0x7fb4ec1005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4ea7fc700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4ec102a90 0x7fb4ec1005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4ea7fc700 1 -- 172.21.15.62:0/2387136663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ec0ff120 msgr2=0x7fb4ec101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4ea7fc700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ec0ff120 0x7fb4ec101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4ea7fc700 1 -- 172.21.15.62:0/2387136663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e4009d00 con 0x7fb4ec1020f0 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.876+0000 7fb4eaffd700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ec0ff120 0x7fb4ec101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.877+0000 7fb4ea7fc700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4ec1020f0 0x7fb4ec1000a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00e9a0 tx=0x7fb4dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.877+0000 7fb4f0962700 1 -- 172.21.15.62:0/2387136663 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc009790 con 0x7fb4ec1020f0 2024-05-08T22:34:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.877+0000 7fb4f0962700 1 -- 172.21.15.62:0/2387136663 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4dc00cb70 con 0x7fb4ec1020f0 2024-05-08T22:34:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.877+0000 7fb4f1964700 1 -- 172.21.15.62:0/2387136663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ec1a11b0 con 0x7fb4ec1020f0 2024-05-08T22:34:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.877+0000 7fb4f0962700 1 -- 172.21.15.62:0/2387136663 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc010ea0 con 0x7fb4ec1020f0 2024-05-08T22:34:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.877+0000 7fb4f1964700 1 -- 172.21.15.62:0/2387136663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ec1a16c0 con 0x7fb4ec1020f0 2024-05-08T22:34:43.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.878+0000 7fb4f1964700 1 -- 172.21.15.62:0/2387136663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ec108330 con 0x7fb4ec1020f0 2024-05-08T22:34:43.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.879+0000 7fb4f0962700 1 -- 172.21.15.62:0/2387136663 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb4dc00cce0 con 0x7fb4ec1020f0 2024-05-08T22:34:43.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.879+0000 7fb4f0962700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb4d80a1120 0x7fb4d80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:43.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.879+0000 7fb4eaffd700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb4d80a1120 0x7fb4d80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:43.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.879+0000 7fb4f0962700 1 -- 172.21.15.62:0/2387136663 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb4dc014070 con 0x7fb4ec1020f0 2024-05-08T22:34:43.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.880+0000 7fb4eaffd700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb4d80a1120 0x7fb4d80a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4005fd0 tx=0x7fb4d4005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:43.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:43.884+0000 7fb4f0962700 1 -- 172.21.15.62:0/2387136663 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb4dc052d60 con 0x7fb4ec1020f0 2024-05-08T22:34:43.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:43 smithi062 ceph-mon[42571]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:43.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:43 smithi062 ceph-mon[42571]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:34:43.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:43 smithi062 ceph-mon[42571]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-05-08T22:34:43.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:43.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:43.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:43.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:43.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:43.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:43.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:43.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:43 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:43.965 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:43 smithi079 ceph-mon[46738]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:43.966 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:43 smithi079 ceph-mon[46738]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:34:43.966 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:43 smithi079 ceph-mon[46738]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-05-08T22:34:43.966 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:43.966 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:43.966 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:43.966 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:43.966 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:43.967 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:43.967 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:43.967 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:43 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:44.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:43 smithi066 ceph-mon[46830]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:44.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:43 smithi066 ceph-mon[46830]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:34:44.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:43 smithi066 ceph-mon[46830]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-05-08T22:34:44.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:44.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:34:44.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:44.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:34:44.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:44.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:44.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:44.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:43 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:44.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.092+0000 7fb4f1964700 1 -- 172.21.15.62:0/2387136663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4ec10c8a0 con 0x7fb4ec1020f0 2024-05-08T22:34:44.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.092+0000 7fb4f0962700 1 -- 172.21.15.62:0/2387136663 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4dc0962d0 con 0x7fb4ec1020f0 2024-05-08T22:34:44.092 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-05-08T22:34:44.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 -- 172.21.15.62:0/2387136663 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb4d80a1120 msgr2=0x7fb4d80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb4d80a1120 0x7fb4d80a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4005fd0 tx=0x7fb4d4005f00 comp rx=0 tx=0).stop 2024-05-08T22:34:44.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 -- 172.21.15.62:0/2387136663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4ec1020f0 msgr2=0x7fb4ec1000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4ec1020f0 0x7fb4ec1000a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00e9a0 tx=0x7fb4dc00ecb0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 -- 172.21.15.62:0/2387136663 shutdown_connections 2024-05-08T22:34:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4ec102a90 0x7fb4ec1005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb4d80a1120 0x7fb4d80a35e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4ec0ff120 0x7fb4ec101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 --2- 172.21.15.62:0/2387136663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4ec1020f0 0x7fb4ec1000a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 -- 172.21.15.62:0/2387136663 >> 172.21.15.62:0/2387136663 conn(0x7fb4ec0faa50 msgr2=0x7fb4ec105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 -- 172.21.15.62:0/2387136663 shutdown_connections 2024-05-08T22:34:44.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.094+0000 7fb4f1964700 1 -- 172.21.15.62:0/2387136663 wait complete. 2024-05-08T22:34:44.107 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:44.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.233+0000 7f5e5c59d700 1 -- 172.21.15.62:0/477591251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e541042b0 msgr2=0x7f5e541066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.233+0000 7f5e5c59d700 1 --2- 172.21.15.62:0/477591251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e541042b0 0x7f5e541066a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009990 tx=0x7f5e44009ca0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.233+0000 7f5e5c59d700 1 -- 172.21.15.62:0/477591251 shutdown_connections 2024-05-08T22:34:44.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.233+0000 7f5e5c59d700 1 --2- 172.21.15.62:0/477591251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e54106be0 0x7f5e54108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.233+0000 7f5e5c59d700 1 --2- 172.21.15.62:0/477591251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e54101980 0x7f5e54103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.233+0000 7f5e5c59d700 1 --2- 172.21.15.62:0/477591251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e541042b0 0x7f5e541066a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.233+0000 7f5e5c59d700 1 -- 172.21.15.62:0/477591251 >> 172.21.15.62:0/477591251 conn(0x7f5e540fb2f0 msgr2=0x7f5e540fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.233+0000 7f5e5c59d700 1 -- 172.21.15.62:0/477591251 shutdown_connections 2024-05-08T22:34:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.233+0000 7f5e5c59d700 1 -- 172.21.15.62:0/477591251 wait complete. 2024-05-08T22:34:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5c59d700 1 Processor -- start 2024-05-08T22:34:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5c59d700 1 -- start start 2024-05-08T22:34:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5c59d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e54101980 0x7f5e54196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5c59d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e541042b0 0x7f5e54197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5c59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54106be0 0x7f5e5419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5c59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5419e950 con 0x7f5e54106be0 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5c59d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5419ea90 con 0x7f5e54101980 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5c59d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5419ec20 con 0x7f5e541042b0 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5ab3a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54106be0 0x7f5e5419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5ab3a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54106be0 0x7f5e5419e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34826/0 (socket says 172.21.15.62:34826) 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5ab3a700 1 -- 172.21.15.62:0/1193468457 learned_addr learned my addr 172.21.15.62:0/1193468457 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e59b38700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e541042b0 0x7f5e54197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.234+0000 7f5e5a339700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e54101980 0x7f5e54196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.235+0000 7f5e5ab3a700 1 -- 172.21.15.62:0/1193468457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e541042b0 msgr2=0x7f5e54197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.235+0000 7f5e5ab3a700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e541042b0 0x7f5e54197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.235+0000 7f5e5ab3a700 1 -- 172.21.15.62:0/1193468457 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e54101980 msgr2=0x7f5e54196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.235+0000 7f5e5ab3a700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e54101980 0x7f5e54196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.235+0000 7f5e5ab3a700 1 -- 172.21.15.62:0/1193468457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e440095f0 con 0x7f5e54106be0 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.235+0000 7f5e5ab3a700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54106be0 0x7f5e5419e380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00ada0 tx=0x7f5e4c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.235+0000 7f5e4b7fe700 1 -- 172.21.15.62:0/1193468457 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00f070 con 0x7f5e54106be0 2024-05-08T22:34:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.235+0000 7f5e4b7fe700 1 -- 172.21.15.62:0/1193468457 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e4c007b30 con 0x7f5e54106be0 2024-05-08T22:34:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.235+0000 7f5e5c59d700 1 -- 172.21.15.62:0/1193468457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5419ef00 con 0x7f5e54106be0 2024-05-08T22:34:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.235+0000 7f5e4b7fe700 1 -- 172.21.15.62:0/1193468457 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c005280 con 0x7f5e54106be0 2024-05-08T22:34:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.235+0000 7f5e5c59d700 1 -- 172.21.15.62:0/1193468457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5419f3d0 con 0x7f5e54106be0 2024-05-08T22:34:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.236+0000 7f5e5c59d700 1 -- 172.21.15.62:0/1193468457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e540fce70 con 0x7f5e54106be0 2024-05-08T22:34:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.237+0000 7f5e4b7fe700 1 -- 172.21.15.62:0/1193468457 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5e4c00a090 con 0x7f5e54106be0 2024-05-08T22:34:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.238+0000 7f5e4b7fe700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e400a11a0 0x7f5e400a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.238+0000 7f5e5a339700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e400a11a0 0x7f5e400a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.238+0000 7f5e4b7fe700 1 -- 172.21.15.62:0/1193468457 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5e4c0cbe00 con 0x7f5e54106be0 2024-05-08T22:34:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.238+0000 7f5e5a339700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e400a11a0 0x7f5e400a3660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5e50007c60 tx=0x7f5e500073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.242+0000 7f5e4b7fe700 1 -- 172.21.15.62:0/1193468457 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5e4c095ac0 con 0x7f5e54106be0 2024-05-08T22:34:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.500+0000 7f5e5c59d700 1 -- 172.21.15.62:0/1193468457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e54067480 con 0x7f5e54106be0 2024-05-08T22:34:44.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.501+0000 7f5e4b7fe700 1 -- 172.21.15.62:0/1193468457 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f5e4c0990e0 con 0x7f5e54106be0 2024-05-08T22:34:44.501 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:44.501 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-08T22:34:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 -- 172.21.15.62:0/1193468457 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e400a11a0 msgr2=0x7f5e400a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e400a11a0 0x7f5e400a3660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5e50007c60 tx=0x7f5e500073d0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 -- 172.21.15.62:0/1193468457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54106be0 msgr2=0x7f5e5419e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54106be0 0x7f5e5419e380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00ada0 tx=0x7f5e4c00c330 comp rx=0 tx=0).stop 2024-05-08T22:34:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 -- 172.21.15.62:0/1193468457 shutdown_connections 2024-05-08T22:34:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54106be0 0x7f5e5419e380 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e400a11a0 0x7f5e400a3660 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e54101980 0x7f5e54196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 --2- 172.21.15.62:0/1193468457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e541042b0 0x7f5e54197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 -- 172.21.15.62:0/1193468457 >> 172.21.15.62:0/1193468457 conn(0x7f5e540fb2f0 msgr2=0x7f5e54102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 -- 172.21.15.62:0/1193468457 shutdown_connections 2024-05-08T22:34:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:44.503+0000 7f5e5c59d700 1 -- 172.21.15.62:0/1193468457 wait complete. 2024-05-08T22:34:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 1 2024-05-08T22:34:44.888 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 08 22:34:44 smithi079 bash[84123]: f3df24c9f3a018ba75f41dd03b52ab737bbefe24a8e2bf8a112fed482d81c8e3 2024-05-08T22:34:44.888 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 08 22:34:44 smithi079 systemd[1]: Started Ceph mds.l for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:34:44.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:44 smithi079 ceph-mon[46738]: Deploying daemon mds.l on smithi079 2024-05-08T22:34:44.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:44 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/296609234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:34:44.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:44 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2387136663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:34:44.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:44 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1193468457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:44.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:44 smithi062 ceph-mon[42571]: Deploying daemon mds.l on smithi079 2024-05-08T22:34:44.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:44 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/296609234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:34:44.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:44 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2387136663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:34:44.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:44 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1193468457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:45.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:44 smithi066 ceph-mon[46830]: Deploying daemon mds.l on smithi079 2024-05-08T22:34:45.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:44 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/296609234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:34:45.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:44 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2387136663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:34:45.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:44 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1193468457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:45.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:45 smithi066 ceph-mon[46830]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:45.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:45 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.79:6832/1379725080,v1:172.21.15.79:6833/1379725080] up:boot 2024-05-08T22:34:45.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:45 smithi066 ceph-mon[46830]: fsmap 1 up:standby 2024-05-08T22:34:45.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:34:45.914 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:45.914 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:45.914 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:45.914 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:45.914 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:45.914 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:45 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:45.914 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:45 smithi066 ceph-mon[46830]: Deploying daemon mds.k on smithi066 2024-05-08T22:34:46.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:45 smithi079 ceph-mon[46738]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:45 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.79:6832/1379725080,v1:172.21.15.79:6833/1379725080] up:boot 2024-05-08T22:34:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:45 smithi079 ceph-mon[46738]: fsmap 1 up:standby 2024-05-08T22:34:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:34:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:45 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:46.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:45 smithi079 ceph-mon[46738]: Deploying daemon mds.k on smithi066 2024-05-08T22:34:46.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:45 smithi062 ceph-mon[42571]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:45 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.79:6832/1379725080,v1:172.21.15.79:6833/1379725080] up:boot 2024-05-08T22:34:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:45 smithi062 ceph-mon[42571]: fsmap 1 up:standby 2024-05-08T22:34:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:34:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:45 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:45 smithi062 ceph-mon[42571]: Deploying daemon mds.k on smithi066 2024-05-08T22:34:46.188 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:34:46 smithi066 systemd[1]: Starting Ceph mds.k for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:34:46.516 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:46.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.658+0000 7f9555e42700 1 -- 172.21.15.62:0/1405862850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9550100f70 msgr2=0x7f95501013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.658+0000 7f9555e42700 1 --2- 172.21.15.62:0/1405862850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9550100f70 0x7f95501013d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f95440099c0 tx=0x7f9544009cd0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.658+0000 7f9555e42700 1 -- 172.21.15.62:0/1405862850 shutdown_connections 2024-05-08T22:34:46.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.658+0000 7f9555e42700 1 --2- 172.21.15.62:0/1405862850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9550101910 0x7f95501081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.658+0000 7f9555e42700 1 --2- 172.21.15.62:0/1405862850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95500ffd80 0x7f95501001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.658+0000 7f9555e42700 1 --2- 172.21.15.62:0/1405862850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9550100f70 0x7f95501013d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.658+0000 7f9555e42700 1 -- 172.21.15.62:0/1405862850 >> 172.21.15.62:0/1405862850 conn(0x7f95500fb310 msgr2=0x7f95500fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:46.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.658+0000 7f9555e42700 1 -- 172.21.15.62:0/1405862850 shutdown_connections 2024-05-08T22:34:46.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.658+0000 7f9555e42700 1 -- 172.21.15.62:0/1405862850 wait complete. 2024-05-08T22:34:46.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.658+0000 7f9555e42700 1 Processor -- start 2024-05-08T22:34:46.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f9555e42700 1 -- start start 2024-05-08T22:34:46.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f9555e42700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95500ffd80 0x7f955010c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:46.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f9555e42700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9550100f70 0x7f955010aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:46.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f9555e42700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9550101910 0x7f955010af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:46.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f9555e42700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955010b710 con 0x7f9550100f70 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f9555e42700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955010b880 con 0x7f9550101910 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f9555e42700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95501a0ea0 con 0x7f95500ffd80 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9550100f70 0x7f955010aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9550100f70 0x7f955010aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54010/0 (socket says 172.21.15.62:54010) 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954effd700 1 -- 172.21.15.62:0/1963786261 learned_addr learned my addr 172.21.15.62:0/1963786261 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954ffff700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9550101910 0x7f955010af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954effd700 1 -- 172.21.15.62:0/1963786261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95500ffd80 msgr2=0x7f955010c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954f7fe700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95500ffd80 0x7f955010c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954effd700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95500ffd80 0x7f955010c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954effd700 1 -- 172.21.15.62:0/1963786261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9550101910 msgr2=0x7f955010af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954effd700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9550101910 0x7f955010af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954effd700 1 -- 172.21.15.62:0/1963786261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440095f0 con 0x7f9550100f70 2024-05-08T22:34:46.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954ffff700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9550101910 0x7f955010af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954f7fe700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95500ffd80 0x7f955010c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.659+0000 7f954effd700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9550100f70 0x7f955010aa30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f954400c010 tx=0x7f9544005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.660+0000 7f954cff9700 1 -- 172.21.15.62:0/1963786261 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954401d070 con 0x7f9550100f70 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.660+0000 7f954cff9700 1 -- 172.21.15.62:0/1963786261 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9544005510 con 0x7f9550100f70 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.660+0000 7f9555e42700 1 -- 172.21.15.62:0/1963786261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95501a10d0 con 0x7f9550100f70 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.660+0000 7f954cff9700 1 -- 172.21.15.62:0/1963786261 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544004c40 con 0x7f9550100f70 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.660+0000 7f9555e42700 1 -- 172.21.15.62:0/1963786261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95501a1490 con 0x7f9550100f70 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.661+0000 7f9555e42700 1 -- 172.21.15.62:0/1963786261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9550067d80 con 0x7f9550100f70 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.662+0000 7f954cff9700 1 -- 172.21.15.62:0/1963786261 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f954400fbc0 con 0x7f9550100f70 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.662+0000 7f954cff9700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f95380a1120 0x7f95380a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.663+0000 7f954f7fe700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f95380a1120 0x7f95380a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:46.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.663+0000 7f954cff9700 1 -- 172.21.15.62:0/1963786261 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f95440cd790 con 0x7f9550100f70 2024-05-08T22:34:46.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.663+0000 7f954f7fe700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f95380a1120 0x7f95380a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9550100dd0 tx=0x7f9540009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:46.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.666+0000 7f954cff9700 1 -- 172.21.15.62:0/1963786261 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f95440974d0 con 0x7f9550100f70 2024-05-08T22:34:46.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.935+0000 7f9555e42700 1 -- 172.21.15.62:0/1963786261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9550067d80 con 0x7f9550100f70 2024-05-08T22:34:46.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.936+0000 7f954cff9700 1 -- 172.21.15.62:0/1963786261 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2014 (secure 0 0 0) 0x7f9550067d80 con 0x7f9550100f70 2024-05-08T22:34:46.936 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:46.936 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 -- 172.21.15.62:0/1963786261 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f95380a1120 msgr2=0x7f95380a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f95380a1120 0x7f95380a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9550100dd0 tx=0x7f9540009260 comp rx=0 tx=0).stop 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 -- 172.21.15.62:0/1963786261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9550100f70 msgr2=0x7f955010aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9550100f70 0x7f955010aa30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f954400c010 tx=0x7f9544005f10 comp rx=0 tx=0).stop 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 -- 172.21.15.62:0/1963786261 shutdown_connections 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f95380a1120 0x7f95380a35e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9550101910 0x7f955010af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95500ffd80 0x7f955010c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 --2- 172.21.15.62:0/1963786261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9550100f70 0x7f955010aa30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 -- 172.21.15.62:0/1963786261 >> 172.21.15.62:0/1963786261 conn(0x7f95500fb310 msgr2=0x7f9550104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 -- 172.21.15.62:0/1963786261 shutdown_connections 2024-05-08T22:34:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:46.938+0000 7f9555e42700 1 -- 172.21.15.62:0/1963786261 wait complete. 2024-05-08T22:34:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 3 2024-05-08T22:34:47.073 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:34:46 smithi066 ceph-mds[82604]: -- 172.21.15.66:0/2247395424 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5610a30a21a0 con 0x5610a3078800 2024-05-08T22:34:47.074 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:34:46 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-k[82581]: starting mds.k at 2024-05-08T22:34:47.074 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:34:46 smithi066 bash[82566]: 7ef6f6162a4bd9bce12fc9f56c81ecfc7c285eb5b5aa9b3dd04c45e16220e0ba 2024-05-08T22:34:47.074 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:34:46 smithi066 systemd[1]: Started Ceph mds.k for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:34:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.66:6832/4065089721,v1:172.21.15.66:6833/4065089721] up:boot 2024-05-08T22:34:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: fsmap 2 up:standby 2024-05-08T22:34:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:34:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:47.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: Deploying daemon mds.j on smithi062 2024-05-08T22:34:47.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:47 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1963786261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:48.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:48.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.66:6832/4065089721,v1:172.21.15.66:6833/4065089721] up:boot 2024-05-08T22:34:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: fsmap 2 up:standby 2024-05-08T22:34:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:34:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: Deploying daemon mds.j on smithi062 2024-05-08T22:34:48.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:47 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1963786261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.66:6832/4065089721,v1:172.21.15.66:6833/4065089721] up:boot 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: fsmap 2 up:standby 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: Deploying daemon mds.j on smithi062 2024-05-08T22:34:48.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:47 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1963786261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:48.330 INFO:journalctl@ceph.mds.j.smithi062.stdout:May 08 22:34:48 smithi062 systemd[1]: Starting Ceph mds.j for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:34:48.921 INFO:journalctl@ceph.mds.j.smithi062.stdout:May 08 22:34:48 smithi062 ceph-mds[94655]: -- 172.21.15.62:0/2495854969 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f55d2421a0 con 0x55f55d219000 2024-05-08T22:34:48.922 INFO:journalctl@ceph.mds.j.smithi062.stdout:May 08 22:34:48 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-j[94632]: starting mds.j at 2024-05-08T22:34:48.922 INFO:journalctl@ceph.mds.j.smithi062.stdout:May 08 22:34:48 smithi062 bash[94617]: 1df86215d157aa21e3160ed7277ddf84badde05c93b94512604feaac07bdf647 2024-05-08T22:34:48.922 INFO:journalctl@ceph.mds.j.smithi062.stdout:May 08 22:34:48 smithi062 systemd[1]: Started Ceph mds.j for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:34:48.950 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:49.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.087+0000 7f8ec00b1700 1 -- 172.21.15.62:0/2436252032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eb8101910 msgr2=0x7f8eb81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:49.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.087+0000 7f8ec00b1700 1 --2- 172.21.15.62:0/2436252032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eb8101910 0x7f8eb81081c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000b0d0 tx=0x7f8eb000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:34:49.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.087+0000 7f8ec00b1700 1 -- 172.21.15.62:0/2436252032 shutdown_connections 2024-05-08T22:34:49.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.087+0000 7f8ec00b1700 1 --2- 172.21.15.62:0/2436252032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eb8101910 0x7f8eb81081c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:49.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.087+0000 7f8ec00b1700 1 --2- 172.21.15.62:0/2436252032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8eb80ffd80 0x7f8eb81001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:49.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.087+0000 7f8ec00b1700 1 --2- 172.21.15.62:0/2436252032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb8100f70 0x7f8eb81013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:49.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.087+0000 7f8ec00b1700 1 -- 172.21.15.62:0/2436252032 >> 172.21.15.62:0/2436252032 conn(0x7f8eb80fb2f0 msgr2=0x7f8eb80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:49.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.087+0000 7f8ec00b1700 1 -- 172.21.15.62:0/2436252032 shutdown_connections 2024-05-08T22:34:49.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.087+0000 7f8ec00b1700 1 -- 172.21.15.62:0/2436252032 wait complete. 2024-05-08T22:34:49.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ec00b1700 1 Processor -- start 2024-05-08T22:34:49.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ec00b1700 1 -- start start 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ec00b1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb80ffd80 0x7f8eb8069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ec00b1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8eb8100f70 0x7f8eb819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ec00b1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eb8101910 0x7f8eb819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ec00b1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb819e9d0 con 0x7f8eb8101910 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ec00b1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb819eb40 con 0x7f8eb80ffd80 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ec00b1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb819ed00 con 0x7f8eb8100f70 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ebe64e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eb8101910 0x7f8eb819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ebe64e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eb8101910 0x7f8eb819e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54038/0 (socket says 172.21.15.62:54038) 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ebe64e700 1 -- 172.21.15.62:0/19735150 learned_addr learned my addr 172.21.15.62:0/19735150 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ebd64c700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8eb8100f70 0x7f8eb819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ebde4d700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb80ffd80 0x7f8eb8069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ebe64e700 1 -- 172.21.15.62:0/19735150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8eb8100f70 msgr2=0x7f8eb819bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:49.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.088+0000 7f8ebe64e700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8eb8100f70 0x7f8eb819bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:49.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.089+0000 7f8ebe64e700 1 -- 172.21.15.62:0/19735150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb80ffd80 msgr2=0x7f8eb8069b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:49.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.089+0000 7f8ebe64e700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb80ffd80 0x7f8eb8069b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:49.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.089+0000 7f8ebe64e700 1 -- 172.21.15.62:0/19735150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb0009d00 con 0x7f8eb8101910 2024-05-08T22:34:49.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.089+0000 7f8ebde4d700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb80ffd80 0x7f8eb8069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:34:49.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.089+0000 7f8ebe64e700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eb8101910 0x7f8eb819e230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0007b60 tx=0x7f8eb0008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:49.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.089+0000 7f8eaeffd700 1 -- 172.21.15.62:0/19735150 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb000e040 con 0x7f8eb8101910 2024-05-08T22:34:49.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.089+0000 7f8eaeffd700 1 -- 172.21.15.62:0/19735150 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eb000be70 con 0x7f8eb8101910 2024-05-08T22:34:49.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.089+0000 7f8ec00b1700 1 -- 172.21.15.62:0/19735150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb819ef80 con 0x7f8eb8101910 2024-05-08T22:34:49.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.089+0000 7f8eaeffd700 1 -- 172.21.15.62:0/19735150 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb001bb00 con 0x7f8eb8101910 2024-05-08T22:34:49.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.089+0000 7f8ec00b1700 1 -- 172.21.15.62:0/19735150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb819f400 con 0x7f8eb8101910 2024-05-08T22:34:49.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.090+0000 7f8ec00b1700 1 -- 172.21.15.62:0/19735150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb8107130 con 0x7f8eb8101910 2024-05-08T22:34:49.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.091+0000 7f8eaeffd700 1 -- 172.21.15.62:0/19735150 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8eb001d670 con 0x7f8eb8101910 2024-05-08T22:34:49.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.091+0000 7f8eaeffd700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8ea40a11a0 0x7f8ea40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:49.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.092+0000 7f8eaeffd700 1 -- 172.21.15.62:0/19735150 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8eb00ce0b0 con 0x7f8eb8101910 2024-05-08T22:34:49.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.092+0000 7f8ebde4d700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8ea40a11a0 0x7f8ea40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:49.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.092+0000 7f8ebde4d700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8ea40a11a0 0x7f8ea40a3660 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4005fd0 tx=0x7f8eb4005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:49.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.095+0000 7f8eaeffd700 1 -- 172.21.15.62:0/19735150 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8eb0097d70 con 0x7f8eb8101910 2024-05-08T22:34:49.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.358+0000 7f8ec00b1700 1 -- 172.21.15.62:0/19735150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8eb8067480 con 0x7f8eb8101910 2024-05-08T22:34:49.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.359+0000 7f8eaeffd700 1 -- 172.21.15.62:0/19735150 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2771 (secure 0 0 0) 0x7f8eb009b390 con 0x7f8eb8101910 2024-05-08T22:34:49.359 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:49.359 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T22:34:49.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.361+0000 7f8ec00b1700 1 -- 172.21.15.62:0/19735150 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8ea40a11a0 msgr2=0x7f8ea40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:49.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.361+0000 7f8ec00b1700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8ea40a11a0 0x7f8ea40a3660 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4005fd0 tx=0x7f8eb4005f00 comp rx=0 tx=0).stop 2024-05-08T22:34:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.361+0000 7f8ec00b1700 1 -- 172.21.15.62:0/19735150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eb8101910 msgr2=0x7f8eb819e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.361+0000 7f8ec00b1700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eb8101910 0x7f8eb819e230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0007b60 tx=0x7f8eb0008f40 comp rx=0 tx=0).stop 2024-05-08T22:34:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.361+0000 7f8ec00b1700 1 -- 172.21.15.62:0/19735150 shutdown_connections 2024-05-08T22:34:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.361+0000 7f8ec00b1700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eb8101910 0x7f8eb819e230 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.361+0000 7f8ec00b1700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8ea40a11a0 0x7f8ea40a3660 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.361+0000 7f8ec00b1700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb80ffd80 0x7f8eb8069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.361+0000 7f8ec00b1700 1 --2- 172.21.15.62:0/19735150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8eb8100f70 0x7f8eb819bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.361+0000 7f8ec00b1700 1 -- 172.21.15.62:0/19735150 >> 172.21.15.62:0/19735150 conn(0x7f8eb80fb2f0 msgr2=0x7f8eb8104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.362+0000 7f8ec00b1700 1 -- 172.21.15.62:0/19735150 shutdown_connections 2024-05-08T22:34:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:49.362+0000 7f8ec00b1700 1 -- 172.21.15.62:0/19735150 wait complete. 2024-05-08T22:34:49.363 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 4 2024-05-08T22:34:49.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:49.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:49.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:49.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:49.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:49.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:49.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:49.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: Deploying daemon mds.d on smithi062 2024-05-08T22:34:49.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.62:6834/3584665258,v1:172.21.15.62:6835/3584665258] up:boot 2024-05-08T22:34:49.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: fsmap 3 up:standby 2024-05-08T22:34:49.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T22:34:49.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:49 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/19735150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:49.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:49.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:49.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:49.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:49.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:49.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:49.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:49.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: Deploying daemon mds.d on smithi062 2024-05-08T22:34:49.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.62:6834/3584665258,v1:172.21.15.62:6835/3584665258] up:boot 2024-05-08T22:34:49.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: fsmap 3 up:standby 2024-05-08T22:34:49.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T22:34:49.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:49 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/19735150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:49.861 INFO:journalctl@ceph.mds.d.smithi062.stdout:May 08 22:34:49 smithi062 systemd[1]: Starting Ceph mds.d for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:34:49.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:49.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:49.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: Deploying daemon mds.d on smithi062 2024-05-08T22:34:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.62:6834/3584665258,v1:172.21.15.62:6835/3584665258] up:boot 2024-05-08T22:34:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: fsmap 3 up:standby 2024-05-08T22:34:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T22:34:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:49 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/19735150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:50.671 INFO:journalctl@ceph.mds.d.smithi062.stdout:May 08 22:34:50 smithi062 ceph-mds[95045]: -- 172.21.15.62:0/982154426 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5581a10881a0 con 0x5581a105e800 2024-05-08T22:34:50.671 INFO:journalctl@ceph.mds.d.smithi062.stdout:May 08 22:34:50 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-d[95022]: starting mds.d at 2024-05-08T22:34:50.671 INFO:journalctl@ceph.mds.d.smithi062.stdout:May 08 22:34:50 smithi062 bash[95008]: 881a253d9902f81f30f60690bd84a355bbddd344ec8db9c28389c67fa44cbf20 2024-05-08T22:34:50.672 INFO:journalctl@ceph.mds.d.smithi062.stdout:May 08 22:34:50 smithi062 systemd[1]: Started Ceph mds.d for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:34:51.359 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:51 smithi066 ceph-mon[46830]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:51.359 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:51.359 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:51.359 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:51.359 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:51.359 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:51.359 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:51.360 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:51 smithi066 ceph-mon[46830]: Deploying daemon mds.e on smithi066 2024-05-08T22:34:51.360 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:51 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.62:6836/2939636235,v1:172.21.15.62:6837/2939636235] up:boot 2024-05-08T22:34:51.360 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:51 smithi066 ceph-mon[46830]: fsmap 4 up:standby 2024-05-08T22:34:51.360 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:51 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T22:34:51.374 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:51.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.521+0000 7f1c36023700 1 -- 172.21.15.62:0/3644939111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c30101910 msgr2=0x7f1c301081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.521+0000 7f1c36023700 1 --2- 172.21.15.62:0/3644939111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c30101910 0x7f1c301081c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800b0d0 tx=0x7f1c2800b3e0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.521+0000 7f1c36023700 1 -- 172.21.15.62:0/3644939111 shutdown_connections 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.521+0000 7f1c36023700 1 --2- 172.21.15.62:0/3644939111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c30101910 0x7f1c301081c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.521+0000 7f1c36023700 1 --2- 172.21.15.62:0/3644939111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c300ffd80 0x7f1c301001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.521+0000 7f1c36023700 1 --2- 172.21.15.62:0/3644939111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c30100f70 0x7f1c301013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.521+0000 7f1c36023700 1 -- 172.21.15.62:0/3644939111 >> 172.21.15.62:0/3644939111 conn(0x7f1c300fb310 msgr2=0x7f1c300fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.521+0000 7f1c36023700 1 -- 172.21.15.62:0/3644939111 shutdown_connections 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.521+0000 7f1c36023700 1 -- 172.21.15.62:0/3644939111 wait complete. 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.521+0000 7f1c36023700 1 Processor -- start 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c36023700 1 -- start start 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c36023700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c300ffd80 0x7f1c30068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c36023700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c30100f70 0x7f1c3019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c36023700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c30101910 0x7f1c3019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c36023700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3019ea60 con 0x7f1c30100f70 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c36023700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3019ebd0 con 0x7f1c30101910 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c36023700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3019ed90 con 0x7f1c300ffd80 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c30100f70 0x7f1c3019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c30100f70 0x7f1c3019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54066/0 (socket says 172.21.15.62:54066) 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2effd700 1 -- 172.21.15.62:0/3945404924 learned_addr learned my addr 172.21.15.62:0/3945404924 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2f7fe700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c300ffd80 0x7f1c30068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2ffff700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c30101910 0x7f1c3019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2effd700 1 -- 172.21.15.62:0/3945404924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c300ffd80 msgr2=0x7f1c30068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2effd700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c300ffd80 0x7f1c30068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2effd700 1 -- 172.21.15.62:0/3945404924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c30101910 msgr2=0x7f1c3019e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2effd700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c30101910 0x7f1c3019e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2effd700 1 -- 172.21.15.62:0/3945404924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c28009d00 con 0x7f1c30100f70 2024-05-08T22:34:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2ffff700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c30101910 0x7f1c3019e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2effd700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c30100f70 0x7f1c3019bc50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400e9a0 tx=0x7f1c2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.522+0000 7f1c2cff9700 1 -- 172.21.15.62:0/3945404924 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24009790 con 0x7f1c30100f70 2024-05-08T22:34:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.523+0000 7f1c2cff9700 1 -- 172.21.15.62:0/3945404924 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c24010430 con 0x7f1c30100f70 2024-05-08T22:34:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.523+0000 7f1c36023700 1 -- 172.21.15.62:0/3945404924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3019efb0 con 0x7f1c30100f70 2024-05-08T22:34:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.523+0000 7f1c2cff9700 1 -- 172.21.15.62:0/3945404924 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24018680 con 0x7f1c30100f70 2024-05-08T22:34:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.523+0000 7f1c36023700 1 -- 172.21.15.62:0/3945404924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3019f4e0 con 0x7f1c30100f70 2024-05-08T22:34:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.523+0000 7f1c36023700 1 -- 172.21.15.62:0/3945404924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c301071b0 con 0x7f1c30100f70 2024-05-08T22:34:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.525+0000 7f1c2cff9700 1 -- 172.21.15.62:0/3945404924 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1c240105a0 con 0x7f1c30100f70 2024-05-08T22:34:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.525+0000 7f1c2cff9700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f1c180a11a0 0x7f1c180a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.526+0000 7f1c2f7fe700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f1c180a11a0 0x7f1c180a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.526+0000 7f1c2cff9700 1 -- 172.21.15.62:0/3945404924 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1c24014070 con 0x7f1c30100f70 2024-05-08T22:34:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.526+0000 7f1c2f7fe700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f1c180a11a0 0x7f1c180a3660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c30100dd0 tx=0x7f1c20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:51.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.529+0000 7f1c2cff9700 1 -- 172.21.15.62:0/3945404924 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1c24096420 con 0x7f1c30100f70 2024-05-08T22:34:51.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:51 smithi079 ceph-mon[46738]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:51.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:51.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:51.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:51.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:51.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:51.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:51.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:51 smithi079 ceph-mon[46738]: Deploying daemon mds.e on smithi066 2024-05-08T22:34:51.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:51 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.62:6836/2939636235,v1:172.21.15.62:6837/2939636235] up:boot 2024-05-08T22:34:51.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:51 smithi079 ceph-mon[46738]: fsmap 4 up:standby 2024-05-08T22:34:51.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:51 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T22:34:51.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:51 smithi062 ceph-mon[42571]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:51.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:51.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:51.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:51.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:51.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:51.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:51.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:51 smithi062 ceph-mon[42571]: Deploying daemon mds.e on smithi066 2024-05-08T22:34:51.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:51 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.62:6836/2939636235,v1:172.21.15.62:6837/2939636235] up:boot 2024-05-08T22:34:51.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:51 smithi062 ceph-mon[42571]: fsmap 4 up:standby 2024-05-08T22:34:51.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:51 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T22:34:51.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.786+0000 7f1c36023700 1 -- 172.21.15.62:0/3945404924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c30067d10 con 0x7f1c30100f70 2024-05-08T22:34:51.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.786+0000 7f1c2cff9700 1 -- 172.21.15.62:0/3945404924 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3528 (secure 0 0 0) 0x7f1c24026090 con 0x7f1c30100f70 2024-05-08T22:34:51.787 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:51.787 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T22:34:51.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.788+0000 7f1c36023700 1 -- 172.21.15.62:0/3945404924 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f1c180a11a0 msgr2=0x7f1c180a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.788+0000 7f1c36023700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f1c180a11a0 0x7f1c180a3660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c30100dd0 tx=0x7f1c20009260 comp rx=0 tx=0).stop 2024-05-08T22:34:51.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.788+0000 7f1c36023700 1 -- 172.21.15.62:0/3945404924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c30100f70 msgr2=0x7f1c3019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:51.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.788+0000 7f1c36023700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c30100f70 0x7f1c3019bc50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400e9a0 tx=0x7f1c2400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.788+0000 7f1c36023700 1 -- 172.21.15.62:0/3945404924 shutdown_connections 2024-05-08T22:34:51.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.788+0000 7f1c36023700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f1c180a11a0 0x7f1c180a3660 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.788+0000 7f1c36023700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c30101910 0x7f1c3019e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.788+0000 7f1c36023700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c300ffd80 0x7f1c30068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.788+0000 7f1c36023700 1 --2- 172.21.15.62:0/3945404924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c30100f70 0x7f1c3019bc50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:51.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.788+0000 7f1c36023700 1 -- 172.21.15.62:0/3945404924 >> 172.21.15.62:0/3945404924 conn(0x7f1c300fb310 msgr2=0x7f1c301042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:51.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.789+0000 7f1c36023700 1 -- 172.21.15.62:0/3945404924 shutdown_connections 2024-05-08T22:34:51.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:51.789+0000 7f1c36023700 1 -- 172.21.15.62:0/3945404924 wait complete. 2024-05-08T22:34:51.790 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 5 2024-05-08T22:34:52.323 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 08 22:34:52 smithi066 bash[82906]: fd2b386bad2c28371634e809ab78ad93b652eddbcce61fcf07c17b609487414a 2024-05-08T22:34:52.323 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 08 22:34:52 smithi066 systemd[1]: Started Ceph mds.e for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:34:52.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:52 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3945404924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:52.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:52 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:52.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:52 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3945404924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:52.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:52.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:52.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:52.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:52.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:52.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:52 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:52.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:52 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3945404924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:52.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:52.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:52.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:52.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:52.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:52.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:52 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:53.609 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:53 smithi079 ceph-mon[46738]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:53.609 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:53 smithi079 ceph-mon[46738]: Deploying daemon mds.f on smithi079 2024-05-08T22:34:53.610 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:53 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.66:6834/2449812673,v1:172.21.15.66:6835/2449812673] up:boot 2024-05-08T22:34:53.610 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:53 smithi079 ceph-mon[46738]: fsmap 5 up:standby 2024-05-08T22:34:53.610 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:53 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:34:53.610 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:34:53 smithi079 systemd[1]: Starting Ceph mds.f for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:34:53.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:53 smithi062 ceph-mon[42571]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:53.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:53 smithi062 ceph-mon[42571]: Deploying daemon mds.f on smithi079 2024-05-08T22:34:53.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:53 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.66:6834/2449812673,v1:172.21.15.66:6835/2449812673] up:boot 2024-05-08T22:34:53.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:53 smithi062 ceph-mon[42571]: fsmap 5 up:standby 2024-05-08T22:34:53.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:53 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:34:53.801 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:53.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:53 smithi066 ceph-mon[46830]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:53.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:53 smithi066 ceph-mon[46830]: Deploying daemon mds.f on smithi079 2024-05-08T22:34:53.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:53 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.66:6834/2449812673,v1:172.21.15.66:6835/2449812673] up:boot 2024-05-08T22:34:53.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:53 smithi066 ceph-mon[46830]: fsmap 5 up:standby 2024-05-08T22:34:53.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:53 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:34:53.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.943+0000 7f6b9931d700 1 -- 172.21.15.62:0/3432191984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94101910 msgr2=0x7f6b941081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.943+0000 7f6b9931d700 1 --2- 172.21.15.62:0/3432191984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94101910 0x7f6b941081c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00b0d0 tx=0x7f6b8c00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.943+0000 7f6b9931d700 1 -- 172.21.15.62:0/3432191984 shutdown_connections 2024-05-08T22:34:53.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.943+0000 7f6b9931d700 1 --2- 172.21.15.62:0/3432191984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94101910 0x7f6b941081c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.943+0000 7f6b9931d700 1 --2- 172.21.15.62:0/3432191984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b940ffd80 0x7f6b941001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.943+0000 7f6b9931d700 1 --2- 172.21.15.62:0/3432191984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b94100f70 0x7f6b941013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.943+0000 7f6b9931d700 1 -- 172.21.15.62:0/3432191984 >> 172.21.15.62:0/3432191984 conn(0x7f6b940fb310 msgr2=0x7f6b940fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:53.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.943+0000 7f6b9931d700 1 -- 172.21.15.62:0/3432191984 shutdown_connections 2024-05-08T22:34:53.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.943+0000 7f6b9931d700 1 -- 172.21.15.62:0/3432191984 wait complete. 2024-05-08T22:34:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.944+0000 7f6b9931d700 1 Processor -- start 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.944+0000 7f6b9931d700 1 -- start start 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.944+0000 7f6b9931d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b940ffd80 0x7f6b94195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.944+0000 7f6b9931d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94100f70 0x7f6b94196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.944+0000 7f6b9931d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b94101910 0x7f6b9419d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.944+0000 7f6b9931d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9419d8e0 con 0x7f6b94100f70 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.944+0000 7f6b9931d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9419da50 con 0x7f6b940ffd80 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.944+0000 7f6b9931d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9419dc10 con 0x7f6b94101910 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b927fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94100f70 0x7f6b94196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b927fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94100f70 0x7f6b94196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54088/0 (socket says 172.21.15.62:54088) 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b927fc700 1 -- 172.21.15.62:0/3374664245 learned_addr learned my addr 172.21.15.62:0/3374664245 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b937fe700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b94101910 0x7f6b9419d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b92ffd700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b940ffd80 0x7f6b94195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b927fc700 1 -- 172.21.15.62:0/3374664245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b94101910 msgr2=0x7f6b9419d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b927fc700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b94101910 0x7f6b9419d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b927fc700 1 -- 172.21.15.62:0/3374664245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b940ffd80 msgr2=0x7f6b94195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b927fc700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b940ffd80 0x7f6b94195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b927fc700 1 -- 172.21.15.62:0/3374664245 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c009d00 con 0x7f6b94100f70 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b927fc700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94100f70 0x7f6b94196330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00e9a0 tx=0x7f6b7c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b8bfff700 1 -- 172.21.15.62:0/3374664245 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c009790 con 0x7f6b94100f70 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b8bfff700 1 -- 172.21.15.62:0/3374664245 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b7c010430 con 0x7f6b94100f70 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b9931d700 1 -- 172.21.15.62:0/3374664245 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9419def0 con 0x7f6b94100f70 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b8bfff700 1 -- 172.21.15.62:0/3374664245 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c018680 con 0x7f6b94100f70 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.945+0000 7f6b9931d700 1 -- 172.21.15.62:0/3374664245 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9419e310 con 0x7f6b94100f70 2024-05-08T22:34:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.946+0000 7f6b9931d700 1 -- 172.21.15.62:0/3374664245 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b94107130 con 0x7f6b94100f70 2024-05-08T22:34:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.947+0000 7f6b8bfff700 1 -- 172.21.15.62:0/3374664245 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6b7c0105a0 con 0x7f6b94100f70 2024-05-08T22:34:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.948+0000 7f6b8bfff700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b800a1120 0x7f6b800a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.948+0000 7f6b92ffd700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b800a1120 0x7f6b800a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.948+0000 7f6b8bfff700 1 -- 172.21.15.62:0/3374664245 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6b7c014070 con 0x7f6b94100f70 2024-05-08T22:34:53.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.949+0000 7f6b92ffd700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b800a1120 0x7f6b800a35e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6b94197270 tx=0x7f6b84009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:53.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:53.952+0000 7f6b8bfff700 1 -- 172.21.15.62:0/3374664245 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6b7c052d60 con 0x7f6b94100f70 2024-05-08T22:34:54.138 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:34:53 smithi079 ceph-mds[84510]: -- 172.21.15.79:0/3886457394 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563f530241a0 con 0x563f52ffb000 2024-05-08T22:34:54.138 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:34:53 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-f[84487]: starting mds.f at 2024-05-08T22:34:54.138 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:34:53 smithi079 bash[84472]: 23c14ee32dc5b9da728508835fda464198e9be46277d90005fbd76c231cf9472 2024-05-08T22:34:54.139 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:34:53 smithi079 systemd[1]: Started Ceph mds.f for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:34:54.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.210+0000 7f6b9931d700 1 -- 172.21.15.62:0/3374664245 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b94068160 con 0x7f6b94100f70 2024-05-08T22:34:54.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.210+0000 7f6b8bfff700 1 -- 172.21.15.62:0/3374664245 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4285 (secure 0 0 0) 0x7f6b7c096230 con 0x7f6b94100f70 2024-05-08T22:34:54.210 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:54.211 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-05-08T22:34:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.212+0000 7f6b9931d700 1 -- 172.21.15.62:0/3374664245 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b800a1120 msgr2=0x7f6b800a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.212+0000 7f6b9931d700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b800a1120 0x7f6b800a35e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6b94197270 tx=0x7f6b84009260 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.212+0000 7f6b9931d700 1 -- 172.21.15.62:0/3374664245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94100f70 msgr2=0x7f6b94196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.212+0000 7f6b9931d700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94100f70 0x7f6b94196330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00e9a0 tx=0x7f6b7c00ecb0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.213+0000 7f6b9931d700 1 -- 172.21.15.62:0/3374664245 shutdown_connections 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.213+0000 7f6b9931d700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b94101910 0x7f6b9419d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.213+0000 7f6b9931d700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b800a1120 0x7f6b800a35e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.213+0000 7f6b9931d700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b940ffd80 0x7f6b94195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.213+0000 7f6b9931d700 1 --2- 172.21.15.62:0/3374664245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94100f70 0x7f6b94196330 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.213+0000 7f6b9931d700 1 -- 172.21.15.62:0/3374664245 >> 172.21.15.62:0/3374664245 conn(0x7f6b940fb310 msgr2=0x7f6b94104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.213+0000 7f6b9931d700 1 -- 172.21.15.62:0/3374664245 shutdown_connections 2024-05-08T22:34:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:54.213+0000 7f6b9931d700 1 -- 172.21.15.62:0/3374664245 wait complete. 2024-05-08T22:34:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 6 2024-05-08T22:34:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:54 smithi079 ceph-mon[46738]: Deploying daemon mds.c on smithi079 2024-05-08T22:34:54.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:54 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3374664245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:54.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:54 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.79:6834/2938225588,v1:172.21.15.79:6835/2938225588] up:boot 2024-05-08T22:34:54.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:54 smithi079 ceph-mon[46738]: fsmap 6 up:standby 2024-05-08T22:34:54.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:54 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:34:55.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:55.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:55.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:55.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:55.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:55.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:55.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:54 smithi066 ceph-mon[46830]: Deploying daemon mds.c on smithi079 2024-05-08T22:34:55.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:54 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3374664245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:55.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:54 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.79:6834/2938225588,v1:172.21.15.79:6835/2938225588] up:boot 2024-05-08T22:34:55.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:54 smithi066 ceph-mon[46830]: fsmap 6 up:standby 2024-05-08T22:34:55.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:54 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:34:55.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:55.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:55.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:55.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:55.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:55.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:55.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:54 smithi062 ceph-mon[42571]: Deploying daemon mds.c on smithi079 2024-05-08T22:34:55.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:54 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3374664245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:55.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:54 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.79:6834/2938225588,v1:172.21.15.79:6835/2938225588] up:boot 2024-05-08T22:34:55.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:54 smithi062 ceph-mon[42571]: fsmap 6 up:standby 2024-05-08T22:34:55.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:54 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:34:55.388 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:34:55 smithi079 systemd[1]: Starting Ceph mds.c for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:34:55.811 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:34:55 smithi079 ceph-mds[84858]: -- 172.21.15.79:0/2459124276 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5580df03a1a0 con 0x5580df010800 2024-05-08T22:34:55.812 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:34:55 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-c[84834]: starting mds.c at 2024-05-08T22:34:55.812 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:34:55 smithi079 bash[84819]: 020c8a8911c212f81c8f2e96604959cedfa33535b41dd830b595d77644bc9ead 2024-05-08T22:34:55.812 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:34:55 smithi079 systemd[1]: Started Ceph mds.c for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:34:56.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:55 smithi066 ceph-mon[46830]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:56.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:56.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:56.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:56.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:56.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:56.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:55 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:56.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:55 smithi079 ceph-mon[46738]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:56.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:56.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:56.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:56.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:56.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:56.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:55 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:56.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:55 smithi062 ceph-mon[42571]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:56.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:56.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:56.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:56.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:56.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:56.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:55 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:56.226 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:56.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.367+0000 7f88343da700 1 -- 172.21.15.62:0/3763489679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882c1019a0 msgr2=0x7f882c103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:56.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.367+0000 7f88343da700 1 --2- 172.21.15.62:0/3763489679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882c1019a0 0x7f882c103d90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8828009990 tx=0x7f8828009ca0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.367+0000 7f88343da700 1 -- 172.21.15.62:0/3763489679 shutdown_connections 2024-05-08T22:34:56.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.367+0000 7f88343da700 1 --2- 172.21.15.62:0/3763489679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f882c106c00 0x7f882c108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.367+0000 7f88343da700 1 --2- 172.21.15.62:0/3763489679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882c1019a0 0x7f882c103d90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.367+0000 7f88343da700 1 --2- 172.21.15.62:0/3763489679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f882c1042d0 0x7f882c1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.367+0000 7f88343da700 1 -- 172.21.15.62:0/3763489679 >> 172.21.15.62:0/3763489679 conn(0x7f882c0fb310 msgr2=0x7f882c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:56.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.367+0000 7f88343da700 1 -- 172.21.15.62:0/3763489679 shutdown_connections 2024-05-08T22:34:56.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.367+0000 7f88343da700 1 -- 172.21.15.62:0/3763489679 wait complete. 2024-05-08T22:34:56.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.367+0000 7f88343da700 1 Processor -- start 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.367+0000 7f88343da700 1 -- start start 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f88343da700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f882c1019a0 0x7f882c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f88343da700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f882c1042d0 0x7f882c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f88343da700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882c106c00 0x7f882c19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f88343da700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c19e8e0 con 0x7f882c106c00 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f88343da700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c19ea20 con 0x7f882c1042d0 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f88343da700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c19ebb0 con 0x7f882c1019a0 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8832977700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882c106c00 0x7f882c19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8832977700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882c106c00 0x7f882c19e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54102/0 (socket says 172.21.15.62:54102) 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8832977700 1 -- 172.21.15.62:0/310090941 learned_addr learned my addr 172.21.15.62:0/310090941 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8831975700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f882c1042d0 0x7f882c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8832176700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f882c1019a0 0x7f882c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:56.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8832977700 1 -- 172.21.15.62:0/310090941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f882c1019a0 msgr2=0x7f882c196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:56.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8832977700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f882c1019a0 0x7f882c196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8832977700 1 -- 172.21.15.62:0/310090941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f882c1042d0 msgr2=0x7f882c1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:56.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8832977700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f882c1042d0 0x7f882c1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8832977700 1 -- 172.21.15.62:0/310090941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88280095f0 con 0x7f882c106c00 2024-05-08T22:34:56.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8831975700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f882c1042d0 0x7f882c1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:34:56.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f8832977700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882c106c00 0x7f882c19e310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f882400ef30 tx=0x7f882400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:56.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f88237fe700 1 -- 172.21.15.62:0/310090941 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824018070 con 0x7f882c106c00 2024-05-08T22:34:56.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f88237fe700 1 -- 172.21.15.62:0/310090941 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f882400f040 con 0x7f882c106c00 2024-05-08T22:34:56.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f88237fe700 1 -- 172.21.15.62:0/310090941 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882400cd00 con 0x7f882c106c00 2024-05-08T22:34:56.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f88343da700 1 -- 172.21.15.62:0/310090941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882c19ee30 con 0x7f882c106c00 2024-05-08T22:34:56.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.368+0000 7f88343da700 1 -- 172.21.15.62:0/310090941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882c19f260 con 0x7f882c106c00 2024-05-08T22:34:56.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.369+0000 7f88343da700 1 -- 172.21.15.62:0/310090941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882c0fce90 con 0x7f882c106c00 2024-05-08T22:34:56.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.371+0000 7f88237fe700 1 -- 172.21.15.62:0/310090941 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8824003a00 con 0x7f882c106c00 2024-05-08T22:34:56.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.371+0000 7f88237fe700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f88180a1120 0x7f88180a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:56.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.371+0000 7f88237fe700 1 -- 172.21.15.62:0/310090941 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f88240cc8e0 con 0x7f882c106c00 2024-05-08T22:34:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.371+0000 7f8832176700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f88180a1120 0x7f88180a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.372+0000 7f8832176700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f88180a1120 0x7f88180a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f882800b2e0 tx=0x7f8828005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:56.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.375+0000 7f88237fe700 1 -- 172.21.15.62:0/310090941 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f88240965a0 con 0x7f882c106c00 2024-05-08T22:34:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.629+0000 7f88343da700 1 -- 172.21.15.62:0/310090941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f882c067d60 con 0x7f882c106c00 2024-05-08T22:34:56.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.630+0000 7f88237fe700 1 -- 172.21.15.62:0/310090941 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5796 (secure 0 0 0) 0x7f8824025380 con 0x7f882c106c00 2024-05-08T22:34:56.630 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:56.630 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-08T22:34:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 -- 172.21.15.62:0/310090941 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f88180a1120 msgr2=0x7f88180a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f88180a1120 0x7f88180a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f882800b2e0 tx=0x7f8828005fd0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 -- 172.21.15.62:0/310090941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882c106c00 msgr2=0x7f882c19e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882c106c00 0x7f882c19e310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f882400ef30 tx=0x7f882400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 -- 172.21.15.62:0/310090941 shutdown_connections 2024-05-08T22:34:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882c106c00 0x7f882c19e310 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f882c1019a0 0x7f882c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f88180a1120 0x7f88180a35e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 --2- 172.21.15.62:0/310090941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f882c1042d0 0x7f882c1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 -- 172.21.15.62:0/310090941 >> 172.21.15.62:0/310090941 conn(0x7f882c0fb310 msgr2=0x7f882c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 -- 172.21.15.62:0/310090941 shutdown_connections 2024-05-08T22:34:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:56.632+0000 7f88343da700 1 -- 172.21.15.62:0/310090941 wait complete. 2024-05-08T22:34:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-05-08T22:34:56.874 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:56 smithi066 ceph-mon[46830]: Deploying daemon mds.h on smithi066 2024-05-08T22:34:56.874 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:56 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] up:boot 2024-05-08T22:34:56.874 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:56 smithi066 ceph-mon[46830]: fsmap 7 up:standby 2024-05-08T22:34:56.874 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:56 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:34:56.874 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:56 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/310090941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:56.874 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:34:56 smithi066 systemd[1]: Starting Ceph mds.h for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:34:56.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:56 smithi062 ceph-mon[42571]: Deploying daemon mds.h on smithi066 2024-05-08T22:34:56.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:56 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] up:boot 2024-05-08T22:34:56.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:56 smithi062 ceph-mon[42571]: fsmap 7 up:standby 2024-05-08T22:34:56.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:56 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:34:56.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:56 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/310090941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:57.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:56 smithi079 ceph-mon[46738]: Deploying daemon mds.h on smithi066 2024-05-08T22:34:57.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:56 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] up:boot 2024-05-08T22:34:57.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:56 smithi079 ceph-mon[46738]: fsmap 7 up:standby 2024-05-08T22:34:57.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:56 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:34:57.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:56 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/310090941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:34:57.573 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:34:57 smithi066 ceph-mds[83285]: -- 172.21.15.66:0/563446771 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559c6ac961a0 con 0x559c6ac6d000 2024-05-08T22:34:57.573 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:34:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-h[83262]: starting mds.h at 2024-05-08T22:34:57.573 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:34:57 smithi066 bash[83247]: 052f79d63a14338a94f3ce8940368f482990819a8dddad65088cbd76537c87d1 2024-05-08T22:34:57.574 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:34:57 smithi066 systemd[1]: Started Ceph mds.h for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:34:58.034 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:57 smithi079 ceph-mon[46738]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:58.035 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:58.035 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:58.035 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:58.035 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:58.035 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:58.035 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:57 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:58.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:57 smithi062 ceph-mon[42571]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:58.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:58.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:58.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:58.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:58.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:58.174 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:57 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:58.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:57 smithi066 ceph-mon[46830]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:34:58.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:58.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:58.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:34:58.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:34:58.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:34:58.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:57 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:34:58.645 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:34:58.746 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:34:58 smithi079 systemd[1]: Starting Ceph mds.i for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:34:58.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.786+0000 7fcb54085700 1 -- 172.21.15.62:0/2808745797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c106b60 msgr2=0x7fcb4c108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.786+0000 7fcb54085700 1 --2- 172.21.15.62:0/2808745797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c106b60 0x7fcb4c108fd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400b0a0 tx=0x7fcb4400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.786+0000 7fcb54085700 1 -- 172.21.15.62:0/2808745797 shutdown_connections 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.786+0000 7fcb54085700 1 --2- 172.21.15.62:0/2808745797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c106b60 0x7fcb4c108fd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.786+0000 7fcb54085700 1 --2- 172.21.15.62:0/2808745797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4c101900 0x7fcb4c103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.786+0000 7fcb54085700 1 --2- 172.21.15.62:0/2808745797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb4c104230 0x7fcb4c106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.786+0000 7fcb54085700 1 -- 172.21.15.62:0/2808745797 >> 172.21.15.62:0/2808745797 conn(0x7fcb4c0fb2f0 msgr2=0x7fcb4c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.786+0000 7fcb54085700 1 -- 172.21.15.62:0/2808745797 shutdown_connections 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.786+0000 7fcb54085700 1 -- 172.21.15.62:0/2808745797 wait complete. 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb54085700 1 Processor -- start 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb54085700 1 -- start start 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb54085700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4c101900 0x7fcb4c196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:58.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb54085700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104230 0x7fcb4c1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb54085700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb4c106b60 0x7fcb4c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb54085700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c19e9c0 con 0x7fcb4c104230 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb54085700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c19eb00 con 0x7fcb4c101900 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb54085700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c19ec90 con 0x7fcb4c106b60 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb51620700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104230 0x7fcb4c1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb51620700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104230 0x7fcb4c1973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59382/0 (socket says 172.21.15.62:59382) 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb51620700 1 -- 172.21.15.62:0/2899268646 learned_addr learned my addr 172.21.15.62:0/2899268646 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb51e21700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4c101900 0x7fcb4c196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb52622700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb4c106b60 0x7fcb4c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb51620700 1 -- 172.21.15.62:0/2899268646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb4c106b60 msgr2=0x7fcb4c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb51620700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb4c106b60 0x7fcb4c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb51620700 1 -- 172.21.15.62:0/2899268646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4c101900 msgr2=0x7fcb4c196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb51620700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4c101900 0x7fcb4c196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:58.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb51620700 1 -- 172.21.15.62:0/2899268646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb44009d00 con 0x7fcb4c104230 2024-05-08T22:34:58.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.787+0000 7fcb51e21700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4c101900 0x7fcb4c196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:34:58.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.788+0000 7fcb51620700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104230 0x7fcb4c1973d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800d980 tx=0x7fcb4800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:58.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.788+0000 7fcb3effd700 1 -- 172.21.15.62:0/2899268646 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4800bb60 con 0x7fcb4c104230 2024-05-08T22:34:58.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.788+0000 7fcb3effd700 1 -- 172.21.15.62:0/2899268646 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb48007640 con 0x7fcb4c104230 2024-05-08T22:34:58.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.788+0000 7fcb3effd700 1 -- 172.21.15.62:0/2899268646 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48004e90 con 0x7fcb4c104230 2024-05-08T22:34:58.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.788+0000 7fcb54085700 1 -- 172.21.15.62:0/2899268646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4c19ef70 con 0x7fcb4c104230 2024-05-08T22:34:58.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.788+0000 7fcb54085700 1 -- 172.21.15.62:0/2899268646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4c19f3f0 con 0x7fcb4c104230 2024-05-08T22:34:58.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.789+0000 7fcb54085700 1 -- 172.21.15.62:0/2899268646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4c0fce70 con 0x7fcb4c104230 2024-05-08T22:34:58.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.790+0000 7fcb3effd700 1 -- 172.21.15.62:0/2899268646 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcb4800bcc0 con 0x7fcb4c104230 2024-05-08T22:34:58.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.791+0000 7fcb3effd700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb380a11a0 0x7fcb380a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:34:58.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.791+0000 7fcb51e21700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb380a11a0 0x7fcb380a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:34:58.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.791+0000 7fcb3effd700 1 -- 172.21.15.62:0/2899268646 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcb480cc690 con 0x7fcb4c104230 2024-05-08T22:34:58.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.791+0000 7fcb51e21700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb380a11a0 0x7fcb380a3660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb40005fd0 tx=0x7fcb40005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:34:58.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:58.794+0000 7fcb3effd700 1 -- 172.21.15.62:0/2899268646 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcb480962c0 con 0x7fcb4c104230 2024-05-08T22:34:59.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.054+0000 7fcb54085700 1 -- 172.21.15.62:0/2899268646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb4c0674d0 con 0x7fcb4c104230 2024-05-08T22:34:59.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.055+0000 7fcb3effd700 1 -- 172.21.15.62:0/2899268646 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6553 (secure 0 0 0) 0x7fcb4800de60 con 0x7fcb4c104230 2024-05-08T22:34:59.055 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:34:59.055 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-08T22:34:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 -- 172.21.15.62:0/2899268646 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb380a11a0 msgr2=0x7fcb380a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb380a11a0 0x7fcb380a3660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb40005fd0 tx=0x7fcb40005f60 comp rx=0 tx=0).stop 2024-05-08T22:34:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 -- 172.21.15.62:0/2899268646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104230 msgr2=0x7fcb4c1973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:34:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104230 0x7fcb4c1973d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800d980 tx=0x7fcb4800dd40 comp rx=0 tx=0).stop 2024-05-08T22:34:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 -- 172.21.15.62:0/2899268646 shutdown_connections 2024-05-08T22:34:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb4c106b60 0x7fcb4c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb380a11a0 0x7fcb380a3660 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb4c101900 0x7fcb4c196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 --2- 172.21.15.62:0/2899268646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104230 0x7fcb4c1973d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:34:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 -- 172.21.15.62:0/2899268646 >> 172.21.15.62:0/2899268646 conn(0x7fcb4c0fb2f0 msgr2=0x7fcb4c102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:34:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 -- 172.21.15.62:0/2899268646 shutdown_connections 2024-05-08T22:34:59.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:34:59.057+0000 7fcb54085700 1 -- 172.21.15.62:0/2899268646 wait complete. 2024-05-08T22:34:59.059 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 9 2024-05-08T22:34:59.069 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:58 smithi062 ceph-mon[42571]: Deploying daemon mds.i on smithi079 2024-05-08T22:34:59.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:58 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] up:boot 2024-05-08T22:34:59.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:58 smithi062 ceph-mon[42571]: fsmap 8 up:standby 2024-05-08T22:34:59.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:58 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:34:59.138 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:34:58 smithi079 ceph-mds[85202]: -- 172.21.15.79:0/1970717793 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560ee12e21a0 con 0x560ee12b9000 2024-05-08T22:34:59.138 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:34:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-i[85180]: starting mds.i at 2024-05-08T22:34:59.138 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:34:58 smithi079 bash[85165]: c064756bb18ed83611bac34c68df203112fccda7c2264ba26450d63d533251bd 2024-05-08T22:34:59.139 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:34:58 smithi079 systemd[1]: Started Ceph mds.i for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:34:59.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:58 smithi079 ceph-mon[46738]: Deploying daemon mds.i on smithi079 2024-05-08T22:34:59.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:58 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] up:boot 2024-05-08T22:34:59.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:58 smithi079 ceph-mon[46738]: fsmap 8 up:standby 2024-05-08T22:34:59.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:58 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:34:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:58 smithi066 ceph-mon[46830]: Deploying daemon mds.i on smithi079 2024-05-08T22:34:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:58 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] up:boot 2024-05-08T22:34:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:58 smithi066 ceph-mon[46830]: fsmap 8 up:standby 2024-05-08T22:34:59.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:58 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:35:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:35:00.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:35:00.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:00.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: Deploying daemon mds.g on smithi062 2024-05-08T22:35:00.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2899268646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:00.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] up:boot 2024-05-08T22:35:00.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: fsmap 9 up:standby 2024-05-08T22:35:00.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:34:59 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:35:00.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:00.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:00.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:35:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:35:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: Deploying daemon mds.g on smithi062 2024-05-08T22:35:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2899268646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] up:boot 2024-05-08T22:35:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: fsmap 9 up:standby 2024-05-08T22:35:00.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:34:59 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:35:00.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:00.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:35:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:35:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: Deploying daemon mds.g on smithi062 2024-05-08T22:35:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2899268646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] up:boot 2024-05-08T22:35:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: fsmap 9 up:standby 2024-05-08T22:35:00.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:34:59 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:35:00.443 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:35:00 smithi062 systemd[1]: Starting Ceph mds.g for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:35:00.921 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:35:00 smithi062 ceph-mds[95578]: -- 172.21.15.62:0/108366584 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560841d441a0 con 0x560841d1b000 2024-05-08T22:35:00.922 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:35:00 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-g[95555]: starting mds.g at 2024-05-08T22:35:00.922 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:35:00 smithi062 bash[95540]: 80e9b39879e2ffd5f9ee979d58006e2d4981729e292197aae426e11aa4424c5e 2024-05-08T22:35:00.922 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:35:00 smithi062 systemd[1]: Started Ceph mds.g for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:35:01.071 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.212+0000 7fe97b00a700 1 -- 172.21.15.62:0/4279032613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9741042f0 msgr2=0x7fe9741066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.212+0000 7fe97b00a700 1 --2- 172.21.15.62:0/4279032613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9741042f0 0x7fe9741066e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe960009990 tx=0x7fe960009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.212+0000 7fe97b00a700 1 -- 172.21.15.62:0/4279032613 shutdown_connections 2024-05-08T22:35:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.212+0000 7fe97b00a700 1 --2- 172.21.15.62:0/4279032613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe974106c20 0x7fe974109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.212+0000 7fe97b00a700 1 --2- 172.21.15.62:0/4279032613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9741019c0 0x7fe974103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.212+0000 7fe97b00a700 1 --2- 172.21.15.62:0/4279032613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9741042f0 0x7fe9741066e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.212+0000 7fe97b00a700 1 -- 172.21.15.62:0/4279032613 >> 172.21.15.62:0/4279032613 conn(0x7fe9740fb310 msgr2=0x7fe9740fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe97b00a700 1 -- 172.21.15.62:0/4279032613 shutdown_connections 2024-05-08T22:35:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe97b00a700 1 -- 172.21.15.62:0/4279032613 wait complete. 2024-05-08T22:35:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe97b00a700 1 Processor -- start 2024-05-08T22:35:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe97b00a700 1 -- start start 2024-05-08T22:35:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe97b00a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9741019c0 0x7fe974196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe97b00a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9741042f0 0x7fe9741972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe97b00a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe974106c20 0x7fe97419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe97b00a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97419e860 con 0x7fe974106c20 2024-05-08T22:35:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe97b00a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97419e9d0 con 0x7fe9741042f0 2024-05-08T22:35:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe97b00a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97419eb90 con 0x7fe9741019c0 2024-05-08T22:35:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe9795a7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe974106c20 0x7fe97419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe9795a7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe974106c20 0x7fe97419e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59424/0 (socket says 172.21.15.62:59424) 2024-05-08T22:35:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.213+0000 7fe9795a7700 1 -- 172.21.15.62:0/1074407586 learned_addr learned my addr 172.21.15.62:0/1074407586 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe978da6700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9741019c0 0x7fe974196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe973fff700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9741042f0 0x7fe9741972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe9795a7700 1 -- 172.21.15.62:0/1074407586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9741019c0 msgr2=0x7fe974196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe9795a7700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9741019c0 0x7fe974196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe9795a7700 1 -- 172.21.15.62:0/1074407586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9741042f0 msgr2=0x7fe9741972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe9795a7700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9741042f0 0x7fe9741972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe9795a7700 1 -- 172.21.15.62:0/1074407586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600095f0 con 0x7fe974106c20 2024-05-08T22:35:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe9795a7700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe974106c20 0x7fe97419e290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe96c007d70 tx=0x7fe96c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe971ffb700 1 -- 172.21.15.62:0/1074407586 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00ac90 con 0x7fe974106c20 2024-05-08T22:35:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe971ffb700 1 -- 172.21.15.62:0/1074407586 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96c00adf0 con 0x7fe974106c20 2024-05-08T22:35:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe97b00a700 1 -- 172.21.15.62:0/1074407586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97419ee70 con 0x7fe974106c20 2024-05-08T22:35:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe971ffb700 1 -- 172.21.15.62:0/1074407586 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c0115e0 con 0x7fe974106c20 2024-05-08T22:35:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.214+0000 7fe97b00a700 1 -- 172.21.15.62:0/1074407586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97419f240 con 0x7fe974106c20 2024-05-08T22:35:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.216+0000 7fe97b00a700 1 -- 172.21.15.62:0/1074407586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9740fce90 con 0x7fe974106c20 2024-05-08T22:35:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.216+0000 7fe971ffb700 1 -- 172.21.15.62:0/1074407586 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe96c012420 con 0x7fe974106c20 2024-05-08T22:35:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.217+0000 7fe971ffb700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe9640a1120 0x7fe9640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.217+0000 7fe971ffb700 1 -- 172.21.15.62:0/1074407586 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe96c0cc610 con 0x7fe974106c20 2024-05-08T22:35:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.217+0000 7fe978da6700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe9640a1120 0x7fe9640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.217+0000 7fe978da6700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe9640a1120 0x7fe9640a35e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9741981f0 tx=0x7fe96800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.221+0000 7fe971ffb700 1 -- 172.21.15.62:0/1074407586 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe96c096240 con 0x7fe974106c20 2024-05-08T22:35:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.483+0000 7fe97b00a700 1 -- 172.21.15.62:0/1074407586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe97410c840 con 0x7fe974106c20 2024-05-08T22:35:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.484+0000 7fe971ffb700 1 -- 172.21.15.62:0/1074407586 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8070 (secure 0 0 0) 0x7fe96c099860 con 0x7fe974106c20 2024-05-08T22:35:01.484 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:01.484 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/1756743060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1756743060},{"type":"v1","addr":"172.21.15.62:6839","nonce":1756743060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/3534785027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":3534785027},{"type":"v1","addr":"172.21.15.79:6839","nonce":3534785027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.486+0000 7fe97b00a700 1 -- 172.21.15.62:0/1074407586 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe9640a1120 msgr2=0x7fe9640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.486+0000 7fe97b00a700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe9640a1120 0x7fe9640a35e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9741981f0 tx=0x7fe96800b410 comp rx=0 tx=0).stop 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.486+0000 7fe97b00a700 1 -- 172.21.15.62:0/1074407586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe974106c20 msgr2=0x7fe97419e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.486+0000 7fe97b00a700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe974106c20 0x7fe97419e290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe96c007d70 tx=0x7fe96c00a340 comp rx=0 tx=0).stop 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.486+0000 7fe97b00a700 1 -- 172.21.15.62:0/1074407586 shutdown_connections 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.486+0000 7fe97b00a700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe974106c20 0x7fe97419e290 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.486+0000 7fe97b00a700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9741019c0 0x7fe974196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.486+0000 7fe97b00a700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe9640a1120 0x7fe9640a35e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.486+0000 7fe97b00a700 1 --2- 172.21.15.62:0/1074407586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9741042f0 0x7fe9741972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.486+0000 7fe97b00a700 1 -- 172.21.15.62:0/1074407586 >> 172.21.15.62:0/1074407586 conn(0x7fe9740fb310 msgr2=0x7fe974102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.487+0000 7fe97b00a700 1 -- 172.21.15.62:0/1074407586 shutdown_connections 2024-05-08T22:35:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:01.487+0000 7fe97b00a700 1 -- 172.21.15.62:0/1074407586 wait complete. 2024-05-08T22:35:01.488 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 11 2024-05-08T22:35:01.781 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:01.781 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:01.781 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:01.782 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:01.782 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:35:01.782 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:35:01.782 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:01.782 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: Deploying daemon mds.a on smithi062 2024-05-08T22:35:01.782 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.62:6838/1756743060,v1:172.21.15.62:6839/1756743060] up:boot 2024-05-08T22:35:01.782 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: fsmap 10 up:standby 2024-05-08T22:35:01.782 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:35:01.782 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:01 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1074407586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:02.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:02.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:02.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:02.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:02.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:35:02.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:35:02.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:02.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: Deploying daemon mds.a on smithi062 2024-05-08T22:35:02.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.62:6838/1756743060,v1:172.21.15.62:6839/1756743060] up:boot 2024-05-08T22:35:02.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: fsmap 10 up:standby 2024-05-08T22:35:02.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:35:02.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:01 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1074407586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:02.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:35:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:35:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: Deploying daemon mds.a on smithi062 2024-05-08T22:35:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.62:6838/1756743060,v1:172.21.15.62:6839/1756743060] up:boot 2024-05-08T22:35:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: fsmap 10 up:standby 2024-05-08T22:35:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:35:02.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:01 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1074407586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:02.148 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:35:01 smithi062 systemd[1]: Starting Ceph mds.a for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:35:02.671 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:35:02 smithi062 ceph-mds[95968]: -- 172.21.15.62:0/2122252784 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f4b634e1a0 con 0x55f4b6324800 2024-05-08T22:35:02.671 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:35:02 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-a[95945]: starting mds.a at 2024-05-08T22:35:02.672 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:35:02 smithi062 bash[95930]: dde50c6dc6ae829688224eac6f195dac575e46adef96a6394297f47688c22020 2024-05-08T22:35:02.672 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:35:02 smithi062 systemd[1]: Started Ceph mds.a for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:35:03.399 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:03 smithi066 ceph-mon[46830]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:03.399 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:03.400 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:03.400 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:03.400 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:35:03.400 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:35:03.400 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:03.400 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:03 smithi066 ceph-mon[46830]: Deploying daemon mds.b on smithi066 2024-05-08T22:35:03.400 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:03 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] up:boot 2024-05-08T22:35:03.400 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:03 smithi066 ceph-mon[46830]: fsmap 11 up:standby 2024-05-08T22:35:03.400 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:03 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:35:03.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:03.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:03 smithi079 ceph-mon[46738]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:03.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:03.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:03.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:03.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:35:03.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:35:03.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:03.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:03 smithi079 ceph-mon[46738]: Deploying daemon mds.b on smithi066 2024-05-08T22:35:03.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:03 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] up:boot 2024-05-08T22:35:03.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:03 smithi079 ceph-mon[46738]: fsmap 11 up:standby 2024-05-08T22:35:03.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:03 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:35:03.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.648+0000 7fcfd6f68700 1 -- 172.21.15.62:0/179696376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfd00ffd80 msgr2=0x7fcfd01001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:03.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.648+0000 7fcfd6f68700 1 --2- 172.21.15.62:0/179696376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfd00ffd80 0x7fcfd01001a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40099c0 tx=0x7fcfc4009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:03.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.648+0000 7fcfd6f68700 1 -- 172.21.15.62:0/179696376 shutdown_connections 2024-05-08T22:35:03.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.648+0000 7fcfd6f68700 1 --2- 172.21.15.62:0/179696376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd0101910 0x7fcfd01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:03.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.648+0000 7fcfd6f68700 1 --2- 172.21.15.62:0/179696376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfd00ffd80 0x7fcfd01001a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:03.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.648+0000 7fcfd6f68700 1 --2- 172.21.15.62:0/179696376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfd0100f70 0x7fcfd01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:03.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.648+0000 7fcfd6f68700 1 -- 172.21.15.62:0/179696376 >> 172.21.15.62:0/179696376 conn(0x7fcfd00fb310 msgr2=0x7fcfd00fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:03.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.648+0000 7fcfd6f68700 1 -- 172.21.15.62:0/179696376 shutdown_connections 2024-05-08T22:35:03.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.648+0000 7fcfd6f68700 1 -- 172.21.15.62:0/179696376 wait complete. 2024-05-08T22:35:03.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd6f68700 1 Processor -- start 2024-05-08T22:35:03.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd6f68700 1 -- start start 2024-05-08T22:35:03.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd6f68700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfd00ffd80 0x7fcfd0103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:03.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd6f68700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfd0100f70 0x7fcfd0102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:03.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd6f68700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd0101910 0x7fcfd0102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:03.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd6f68700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0103030 con 0x7fcfd00ffd80 2024-05-08T22:35:03.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd6f68700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd01031a0 con 0x7fcfd0101910 2024-05-08T22:35:03.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd6f68700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd01a0eb0 con 0x7fcfd0100f70 2024-05-08T22:35:03.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd4d04700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfd00ffd80 0x7fcfd0103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:03.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd4d04700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfd00ffd80 0x7fcfd0103cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59454/0 (socket says 172.21.15.62:59454) 2024-05-08T22:35:03.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd4d04700 1 -- 172.21.15.62:0/605013129 learned_addr learned my addr 172.21.15.62:0/605013129 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:03.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd5505700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd0101910 0x7fcfd0102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:03.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfcffff700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfd0100f70 0x7fcfd0102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:03.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd4d04700 1 -- 172.21.15.62:0/605013129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfd0100f70 msgr2=0x7fcfd0102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:03.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd4d04700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfd0100f70 0x7fcfd0102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:03.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd4d04700 1 -- 172.21.15.62:0/605013129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd0101910 msgr2=0x7fcfd0102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:03.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd4d04700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd0101910 0x7fcfd0102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:03.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd4d04700 1 -- 172.21.15.62:0/605013129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc40095f0 con 0x7fcfd00ffd80 2024-05-08T22:35:03.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.649+0000 7fcfd4d04700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfd00ffd80 0x7fcfd0103cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4006010 tx=0x7fcfc400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:03.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.650+0000 7fcfcdffb700 1 -- 172.21.15.62:0/605013129 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc401d070 con 0x7fcfd00ffd80 2024-05-08T22:35:03.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.650+0000 7fcfd6f68700 1 -- 172.21.15.62:0/605013129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd01a10e0 con 0x7fcfd00ffd80 2024-05-08T22:35:03.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.650+0000 7fcfcdffb700 1 -- 172.21.15.62:0/605013129 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfc4005020 con 0x7fcfd00ffd80 2024-05-08T22:35:03.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.650+0000 7fcfcdffb700 1 -- 172.21.15.62:0/605013129 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4004b10 con 0x7fcfd00ffd80 2024-05-08T22:35:03.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.650+0000 7fcfd6f68700 1 -- 172.21.15.62:0/605013129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd01a1450 con 0x7fcfd00ffd80 2024-05-08T22:35:03.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.651+0000 7fcfd6f68700 1 -- 172.21.15.62:0/605013129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd0067d10 con 0x7fcfd00ffd80 2024-05-08T22:35:03.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.652+0000 7fcfcdffb700 1 -- 172.21.15.62:0/605013129 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcfc4005190 con 0x7fcfd00ffd80 2024-05-08T22:35:03.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.652+0000 7fcfcdffb700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcfc00a1120 0x7fcfc00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:03.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.653+0000 7fcfcffff700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcfc00a1120 0x7fcfc00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:03.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.653+0000 7fcfcdffb700 1 -- 172.21.15.62:0/605013129 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcfc40cd0b0 con 0x7fcfd00ffd80 2024-05-08T22:35:03.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.653+0000 7fcfcffff700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcfc00a1120 0x7fcfc00a35e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009c60 tx=0x7fcfbc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:03.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.656+0000 7fcfcdffb700 1 -- 172.21.15.62:0/605013129 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcfc4096d70 con 0x7fcfd00ffd80 2024-05-08T22:35:03.665 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:35:03 smithi066 systemd[1]: Starting Ceph mds.b for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:35:03.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:03 smithi062 ceph-mon[42571]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:03.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:03.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:03.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:03.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T22:35:03.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T22:35:03.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:03.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:03 smithi062 ceph-mon[42571]: Deploying daemon mds.b on smithi066 2024-05-08T22:35:03.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:03 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] up:boot 2024-05-08T22:35:03.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:03 smithi062 ceph-mon[42571]: fsmap 11 up:standby 2024-05-08T22:35:03.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:03 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:35:03.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.914+0000 7fcfd6f68700 1 -- 172.21.15.62:0/605013129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfd010b5e0 con 0x7fcfd00ffd80 2024-05-08T22:35:03.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.915+0000 7fcfcdffb700 1 -- 172.21.15.62:0/605013129 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8825 (secure 0 0 0) 0x7fcfc409a390 con 0x7fcfd00ffd80 2024-05-08T22:35:03.915 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:03.915 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/1756743060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1756743060},{"type":"v1","addr":"172.21.15.62:6839","nonce":1756743060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/780017086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":780017086},{"type":"v1","addr":"172.21.15.62:6841","nonce":780017086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/3534785027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":3534785027},{"type":"v1","addr":"172.21.15.79:6839","nonce":3534785027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-08T22:35:03.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 -- 172.21.15.62:0/605013129 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcfc00a1120 msgr2=0x7fcfc00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:03.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcfc00a1120 0x7fcfc00a35e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009c60 tx=0x7fcfbc009260 comp rx=0 tx=0).stop 2024-05-08T22:35:03.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 -- 172.21.15.62:0/605013129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfd00ffd80 msgr2=0x7fcfd0103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:03.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfd00ffd80 0x7fcfd0103cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4006010 tx=0x7fcfc400f690 comp rx=0 tx=0).stop 2024-05-08T22:35:03.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 -- 172.21.15.62:0/605013129 shutdown_connections 2024-05-08T22:35:03.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcfc00a1120 0x7fcfc00a35e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:03.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcfd0101910 0x7fcfd0102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:03.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfd00ffd80 0x7fcfd0103cc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:03.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 --2- 172.21.15.62:0/605013129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfd0100f70 0x7fcfd0102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:03.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 -- 172.21.15.62:0/605013129 >> 172.21.15.62:0/605013129 conn(0x7fcfd00fb310 msgr2=0x7fcfd0068190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:03.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 -- 172.21.15.62:0/605013129 shutdown_connections 2024-05-08T22:35:03.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:03.917+0000 7fcfd6f68700 1 -- 172.21.15.62:0/605013129 wait complete. 2024-05-08T22:35:03.918 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 12 2024-05-08T22:35:04.323 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:35:04 smithi066 ceph-mds[83627]: -- 172.21.15.66:0/2100959316 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560b798301a0 con 0x560b79806800 2024-05-08T22:35:04.323 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:35:04 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-b[83604]: starting mds.b at 2024-05-08T22:35:04.323 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:35:04 smithi066 bash[83589]: 9099b83c7d1f5a77c0849fd75d1e420dc06500a0fbab9e2202ac871b378c02e9 2024-05-08T22:35:04.324 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:35:04 smithi066 systemd[1]: Started Ceph mds.b for 49d88ea8-0d8a-11ef-bc97-c7b262605968. 2024-05-08T22:35:04.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:04 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/605013129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:04.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:04 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/605013129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:04.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:04 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/605013129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:04.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:04.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:05.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:05 smithi079 ceph-mon[46738]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:05.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:35:05.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:05 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] up:boot 2024-05-08T22:35:05.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:05 smithi079 ceph-mon[46738]: fsmap 12 up:standby 2024-05-08T22:35:05.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:35:05.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:05 smithi062 ceph-mon[42571]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:05.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:35:05.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:05 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] up:boot 2024-05-08T22:35:05.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:05 smithi062 ceph-mon[42571]: fsmap 12 up:standby 2024-05-08T22:35:05.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:35:05.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:05 smithi066 ceph-mon[46830]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:05.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:35:05.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:05 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] up:boot 2024-05-08T22:35:05.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:05 smithi066 ceph-mon[46830]: fsmap 12 up:standby 2024-05-08T22:35:05.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:35:05.931 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.061+0000 7f12945f8700 1 -- 172.21.15.62:0/3087964282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f128c101960 msgr2=0x7f128c103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.061+0000 7f12945f8700 1 --2- 172.21.15.62:0/3087964282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f128c101960 0x7f128c103d50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1288009990 tx=0x7f1288009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.061+0000 7f12945f8700 1 -- 172.21.15.62:0/3087964282 shutdown_connections 2024-05-08T22:35:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.061+0000 7f12945f8700 1 --2- 172.21.15.62:0/3087964282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f128c106bc0 0x7f128c108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.061+0000 7f12945f8700 1 --2- 172.21.15.62:0/3087964282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f128c101960 0x7f128c103d50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.061+0000 7f12945f8700 1 --2- 172.21.15.62:0/3087964282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f128c104290 0x7f128c106680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.061+0000 7f12945f8700 1 -- 172.21.15.62:0/3087964282 >> 172.21.15.62:0/3087964282 conn(0x7f128c0fb310 msgr2=0x7f128c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.061+0000 7f12945f8700 1 -- 172.21.15.62:0/3087964282 shutdown_connections 2024-05-08T22:35:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.061+0000 7f12945f8700 1 -- 172.21.15.62:0/3087964282 wait complete. 2024-05-08T22:35:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f12945f8700 1 Processor -- start 2024-05-08T22:35:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f12945f8700 1 -- start start 2024-05-08T22:35:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f12945f8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f128c101960 0x7f128c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f12945f8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f128c104290 0x7f128c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f12945f8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f128c106bc0 0x7f128c19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f12945f8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c19e8e0 con 0x7f128c106bc0 2024-05-08T22:35:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f12945f8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c19ea20 con 0x7f128c104290 2024-05-08T22:35:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f12945f8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c19ebb0 con 0x7f128c101960 2024-05-08T22:35:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1292b95700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f128c106bc0 0x7f128c19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1292b95700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f128c106bc0 0x7f128c19e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59482/0 (socket says 172.21.15.62:59482) 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1292b95700 1 -- 172.21.15.62:0/1907294161 learned_addr learned my addr 172.21.15.62:0/1907294161 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1292394700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f128c101960 0x7f128c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1291b93700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f128c104290 0x7f128c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1292b95700 1 -- 172.21.15.62:0/1907294161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f128c101960 msgr2=0x7f128c196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1292b95700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f128c101960 0x7f128c196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1292b95700 1 -- 172.21.15.62:0/1907294161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f128c104290 msgr2=0x7f128c1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1292b95700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f128c104290 0x7f128c1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1292b95700 1 -- 172.21.15.62:0/1907294161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12880095f0 con 0x7f128c106bc0 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1291b93700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f128c104290 0x7f128c1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.062+0000 7f1292b95700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f128c106bc0 0x7f128c19e310 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f128400ef30 tx=0x7f128400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.063+0000 7f12837fe700 1 -- 172.21.15.62:0/1907294161 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284018070 con 0x7f128c106bc0 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.063+0000 7f12837fe700 1 -- 172.21.15.62:0/1907294161 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f128400f040 con 0x7f128c106bc0 2024-05-08T22:35:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.063+0000 7f12837fe700 1 -- 172.21.15.62:0/1907294161 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128400cd00 con 0x7f128c106bc0 2024-05-08T22:35:06.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.063+0000 7f12945f8700 1 -- 172.21.15.62:0/1907294161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c19ee30 con 0x7f128c106bc0 2024-05-08T22:35:06.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.063+0000 7f12945f8700 1 -- 172.21.15.62:0/1907294161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c19f260 con 0x7f128c106bc0 2024-05-08T22:35:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.064+0000 7f12945f8700 1 -- 172.21.15.62:0/1907294161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c0fce90 con 0x7f128c106bc0 2024-05-08T22:35:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.065+0000 7f12837fe700 1 -- 172.21.15.62:0/1907294161 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1284003a00 con 0x7f128c106bc0 2024-05-08T22:35:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.066+0000 7f12837fe700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f12780a11a0 0x7f12780a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.066+0000 7f1292394700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f12780a11a0 0x7f12780a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.066+0000 7f12837fe700 1 -- 172.21.15.62:0/1907294161 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f12840cc8e0 con 0x7f128c106bc0 2024-05-08T22:35:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.066+0000 7f1292394700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f12780a11a0 0x7f12780a3660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f128800b2e0 tx=0x7f1288005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.069+0000 7f12837fe700 1 -- 172.21.15.62:0/1907294161 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1284096500 con 0x7f128c106bc0 2024-05-08T22:35:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.333+0000 7f12945f8700 1 -- 172.21.15.62:0/1907294161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f128c067d60 con 0x7f128c106bc0 2024-05-08T22:35:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.334+0000 7f12837fe700 1 -- 172.21.15.62:0/1907294161 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9583 (secure 0 0 0) 0x7f1284099b20 con 0x7f128c106bc0 2024-05-08T22:35:06.334 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:06.334 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/1756743060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1756743060},{"type":"v1","addr":"172.21.15.62:6839","nonce":1756743060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/780017086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":780017086},{"type":"v1","addr":"172.21.15.62:6841","nonce":780017086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/3534785027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":3534785027},{"type":"v1","addr":"172.21.15.79:6839","nonce":3534785027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24533,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/3474672912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":3474672912},{"type":"v1","addr":"172.21.15.66:6839","nonce":3474672912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-05-08T22:35:06.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.336+0000 7f12945f8700 1 -- 172.21.15.62:0/1907294161 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f12780a11a0 msgr2=0x7f12780a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.336+0000 7f12945f8700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f12780a11a0 0x7f12780a3660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f128800b2e0 tx=0x7f1288005fd0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.336+0000 7f12945f8700 1 -- 172.21.15.62:0/1907294161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f128c106bc0 msgr2=0x7f128c19e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.336+0000 7f12945f8700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f128c106bc0 0x7f128c19e310 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f128400ef30 tx=0x7f128400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.336+0000 7f12945f8700 1 -- 172.21.15.62:0/1907294161 shutdown_connections 2024-05-08T22:35:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.336+0000 7f12945f8700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f128c106bc0 0x7f128c19e310 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.336+0000 7f12945f8700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f128c101960 0x7f128c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.336+0000 7f12945f8700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f12780a11a0 0x7f12780a3660 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.336+0000 7f12945f8700 1 --2- 172.21.15.62:0/1907294161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f128c104290 0x7f128c1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.336+0000 7f12945f8700 1 -- 172.21.15.62:0/1907294161 >> 172.21.15.62:0/1907294161 conn(0x7f128c0fb310 msgr2=0x7f128c1037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.336+0000 7f12945f8700 1 -- 172.21.15.62:0/1907294161 shutdown_connections 2024-05-08T22:35:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.337+0000 7f12945f8700 1 -- 172.21.15.62:0/1907294161 wait complete. 2024-05-08T22:35:06.338 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 13 2024-05-08T22:35:06.351 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.484+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/669425693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a880ffd80 msgr2=0x7f7a881001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.484+0000 7f7a8c8f3700 1 --2- 172.21.15.62:0/669425693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a880ffd80 0x7f7a881001a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7a780099c0 tx=0x7f7a78009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/669425693 shutdown_connections 2024-05-08T22:35:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 --2- 172.21.15.62:0/669425693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88101910 0x7f7a881081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 --2- 172.21.15.62:0/669425693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a880ffd80 0x7f7a881001a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 --2- 172.21.15.62:0/669425693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a88100f70 0x7f7a881013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/669425693 >> 172.21.15.62:0/669425693 conn(0x7f7a880fb310 msgr2=0x7f7a880fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/669425693 shutdown_connections 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/669425693 wait complete. 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 Processor -- start 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 -- start start 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a880ffd80 0x7f7a88068c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88100f70 0x7f7a8819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.485+0000 7f7a8c8f3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a88101910 0x7f7a8819e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8c8f3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8819e8c0 con 0x7f7a880ffd80 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8c8f3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8819ea30 con 0x7f7a88101910 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8c8f3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8819ebf0 con 0x7f7a88100f70 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a880ffd80 0x7f7a88068c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a880ffd80 0x7f7a88068c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48212/0 (socket says 172.21.15.62:48212) 2024-05-08T22:35:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8659c700 1 -- 172.21.15.62:0/2247991108 learned_addr learned my addr 172.21.15.62:0/2247991108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a86d9d700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a88101910 0x7f7a8819e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a85d9b700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88100f70 0x7f7a8819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8659c700 1 -- 172.21.15.62:0/2247991108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88100f70 msgr2=0x7f7a8819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8659c700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88100f70 0x7f7a8819bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8659c700 1 -- 172.21.15.62:0/2247991108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a88101910 msgr2=0x7f7a8819e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8659c700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a88101910 0x7f7a8819e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8659c700 1 -- 172.21.15.62:0/2247991108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a780095f0 con 0x7f7a880ffd80 2024-05-08T22:35:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a86d9d700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a88101910 0x7f7a8819e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8659c700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a880ffd80 0x7f7a88068c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7a78003730 tx=0x7f7a7800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a777fe700 1 -- 172.21.15.62:0/2247991108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7801d070 con 0x7f7a880ffd80 2024-05-08T22:35:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a777fe700 1 -- 172.21.15.62:0/2247991108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a780059a0 con 0x7f7a880ffd80 2024-05-08T22:35:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/2247991108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8819ee70 con 0x7f7a880ffd80 2024-05-08T22:35:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a777fe700 1 -- 172.21.15.62:0/2247991108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78004990 con 0x7f7a880ffd80 2024-05-08T22:35:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.486+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/2247991108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8819f240 con 0x7f7a880ffd80 2024-05-08T22:35:06.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.487+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/2247991108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a88067d80 con 0x7f7a880ffd80 2024-05-08T22:35:06.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.489+0000 7f7a777fe700 1 -- 172.21.15.62:0/2247991108 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7a78005210 con 0x7f7a880ffd80 2024-05-08T22:35:06.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.489+0000 7f7a777fe700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a700a1120 0x7f7a700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.490+0000 7f7a85d9b700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a700a1120 0x7f7a700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.490+0000 7f7a777fe700 1 -- 172.21.15.62:0/2247991108 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7a780cd1e0 con 0x7f7a880ffd80 2024-05-08T22:35:06.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.490+0000 7f7a85d9b700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a700a1120 0x7f7a700a35e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7a88069240 tx=0x7f7a7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:06.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.493+0000 7f7a777fe700 1 -- 172.21.15.62:0/2247991108 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7a78096ea0 con 0x7f7a880ffd80 2024-05-08T22:35:06.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:06.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:35:06.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:06 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:35:06.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:06 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1907294161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:06.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:06.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:35:06.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:06 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:35:06.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:06 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1907294161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.713+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/2247991108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a88107250 con 0x7f7a880ffd80 2024-05-08T22:35:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.714+0000 7f7a777fe700 1 -- 172.21.15.62:0/2247991108 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f7a7809a4c0 con 0x7f7a880ffd80 2024-05-08T22:35:06.714 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":75,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:34:35.551157+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/2247991108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a700a1120 msgr2=0x7f7a700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a700a1120 0x7f7a700a35e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7a88069240 tx=0x7f7a7c009260 comp rx=0 tx=0).stop 2024-05-08T22:35:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/2247991108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a880ffd80 msgr2=0x7f7a88068c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a880ffd80 0x7f7a88068c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7a78003730 tx=0x7f7a7800f670 comp rx=0 tx=0).stop 2024-05-08T22:35:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/2247991108 shutdown_connections 2024-05-08T22:35:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a700a1120 0x7f7a700a35e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a88101910 0x7f7a8819e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a880ffd80 0x7f7a88068c80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 --2- 172.21.15.62:0/2247991108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a88100f70 0x7f7a8819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/2247991108 >> 172.21.15.62:0/2247991108 conn(0x7f7a880fb310 msgr2=0x7f7a88104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:06.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/2247991108 shutdown_connections 2024-05-08T22:35:06.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.716+0000 7f7a8c8f3700 1 -- 172.21.15.62:0/2247991108 wait complete. 2024-05-08T22:35:06.729 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:31:54.466249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:35:06.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:35:06.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:35:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:06 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:35:06.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:06 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1907294161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.871+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/1319755876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f08100090 msgr2=0x7f4f08100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.871+0000 7f4f0d9f7700 1 --2- 172.21.15.62:0/1319755876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f08100090 0x7f4f08100510 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0099c0 tx=0x7f4efc009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.871+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/1319755876 shutdown_connections 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.871+0000 7f4f0d9f7700 1 --2- 172.21.15.62:0/1319755876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f08100a50 0x7f4f08100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.871+0000 7f4f0d9f7700 1 --2- 172.21.15.62:0/1319755876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f080ff730 0x7f4f080ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.871+0000 7f4f0d9f7700 1 --2- 172.21.15.62:0/1319755876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f08100090 0x7f4f08100510 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.871+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/1319755876 >> 172.21.15.62:0/1319755876 conn(0x7f4f080fb290 msgr2=0x7f4f080fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.871+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/1319755876 shutdown_connections 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.871+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/1319755876 wait complete. 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.871+0000 7f4f0d9f7700 1 Processor -- start 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f0d9f7700 1 -- start start 2024-05-08T22:35:06.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f0d9f7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f080ff730 0x7f4f08199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f0d9f7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f08100090 0x7f4f08199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f0d9f7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f08100a50 0x7f4f081a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f0d9f7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081a0b20 con 0x7f4f08100090 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f0d9f7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081a0c60 con 0x7f4f080ff730 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f0d9f7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081a0df0 con 0x7f4f08100a50 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f067fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f08100090 0x7f4f08199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f06ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f080ff730 0x7f4f08199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f06ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f080ff730 0x7f4f08199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.62:57732/0 (socket says 172.21.15.62:57732) 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f06ffd700 1 -- 172.21.15.62:0/3467179062 learned_addr learned my addr 172.21.15.62:0/3467179062 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f077fe700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f08100a50 0x7f4f081a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f06ffd700 1 -- 172.21.15.62:0/3467179062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f08100a50 msgr2=0x7f4f081a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f06ffd700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f08100a50 0x7f4f081a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f06ffd700 1 -- 172.21.15.62:0/3467179062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f08100090 msgr2=0x7f4f08199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:06.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f06ffd700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f08100090 0x7f4f08199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:06.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f06ffd700 1 -- 172.21.15.62:0/3467179062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8009520 con 0x7f4f080ff730 2024-05-08T22:35:06.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f067fc700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f08100090 0x7f4f08199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:06.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.872+0000 7f4f06ffd700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f080ff730 0x7f4f08199040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800eb30 tx=0x7f4ef800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:06.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.873+0000 7f4f0c9f5700 1 -- 172.21.15.62:0/3467179062 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8009880 con 0x7f4f080ff730 2024-05-08T22:35:06.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.873+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/3467179062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc0095f0 con 0x7f4f080ff730 2024-05-08T22:35:06.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.873+0000 7f4f0c9f5700 1 -- 172.21.15.62:0/3467179062 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ef80044e0 con 0x7f4f080ff730 2024-05-08T22:35:06.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.873+0000 7f4f0c9f5700 1 -- 172.21.15.62:0/3467179062 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8003d80 con 0x7f4f080ff730 2024-05-08T22:35:06.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.873+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/3467179062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f081a1450 con 0x7f4f080ff730 2024-05-08T22:35:06.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.874+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/3467179062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef4005320 con 0x7f4f080ff730 2024-05-08T22:35:06.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.875+0000 7f4f0c9f5700 1 -- 172.21.15.62:0/3467179062 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4ef8018430 con 0x7f4f080ff730 2024-05-08T22:35:06.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.875+0000 7f4f0c9f5700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4ef00a1120 0x7f4ef00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:06.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.875+0000 7f4f0c9f5700 1 -- 172.21.15.62:0/3467179062 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4ef8014070 con 0x7f4f080ff730 2024-05-08T22:35:06.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.875+0000 7f4f067fc700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4ef00a1120 0x7f4ef00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:06.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.876+0000 7f4f067fc700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4ef00a1120 0x7f4ef00a35e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4efc003730 tx=0x7f4efc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:06.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:06.880+0000 7f4f0c9f5700 1 -- 172.21.15.62:0/3467179062 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4ef8096580 con 0x7f4f080ff730 2024-05-08T22:35:07.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.128+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/3467179062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ef4005b50 con 0x7f4f080ff730 2024-05-08T22:35:07.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.129+0000 7f4f0c9f5700 1 -- 172.21.15.62:0/3467179062 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ef8099ba0 con 0x7f4f080ff730 2024-05-08T22:35:07.129 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-05-08T22:35:07.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/3467179062 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4ef00a1120 msgr2=0x7f4ef00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:07.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4ef00a1120 0x7f4ef00a35e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4efc003730 tx=0x7f4efc01a040 comp rx=0 tx=0).stop 2024-05-08T22:35:07.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/3467179062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f080ff730 msgr2=0x7f4f08199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:07.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f080ff730 0x7f4f08199040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800eb30 tx=0x7f4ef800ee40 comp rx=0 tx=0).stop 2024-05-08T22:35:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/3467179062 shutdown_connections 2024-05-08T22:35:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f08100a50 0x7f4f081a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4ef00a1120 0x7f4ef00a35e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f080ff730 0x7f4f08199040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 --2- 172.21.15.62:0/3467179062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f08100090 0x7f4f08199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/3467179062 >> 172.21.15.62:0/3467179062 conn(0x7f4f080fb290 msgr2=0x7f4f080fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/3467179062 shutdown_connections 2024-05-08T22:35:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.131+0000 7f4f0d9f7700 1 -- 172.21.15.62:0/3467179062 wait complete. 2024-05-08T22:35:07.145 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T22:35:07.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.286+0000 7fa4de40c700 1 -- 172.21.15.62:0/3188604344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d80ff160 msgr2=0x7fa4d80ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:07.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.286+0000 7fa4de40c700 1 --2- 172.21.15.62:0/3188604344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d80ff160 0x7fa4d80ff580 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009990 tx=0x7fa4c8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.286+0000 7fa4de40c700 1 -- 172.21.15.62:0/3188604344 shutdown_connections 2024-05-08T22:35:07.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.286+0000 7fa4de40c700 1 --2- 172.21.15.62:0/3188604344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d8102ad0 0x7fa4d8109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.286+0000 7fa4de40c700 1 --2- 172.21.15.62:0/3188604344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d80ff160 0x7fa4d80ff580 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.286+0000 7fa4de40c700 1 --2- 172.21.15.62:0/3188604344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4d8102130 0x7fa4d8102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.286+0000 7fa4de40c700 1 -- 172.21.15.62:0/3188604344 >> 172.21.15.62:0/3188604344 conn(0x7fa4d80faa90 msgr2=0x7fa4d80fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.286+0000 7fa4de40c700 1 -- 172.21.15.62:0/3188604344 shutdown_connections 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.286+0000 7fa4de40c700 1 -- 172.21.15.62:0/3188604344 wait complete. 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.286+0000 7fa4de40c700 1 Processor -- start 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.286+0000 7fa4de40c700 1 -- start start 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4de40c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d80ff160 0x7fa4d8196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4de40c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4d8102130 0x7fa4d8197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4de40c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d8102ad0 0x7fa4d819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4de40c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d819e950 con 0x7fa4d8102ad0 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4de40c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d819ea90 con 0x7fa4d8102130 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4de40c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d819ec20 con 0x7fa4d80ff160 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4dc9a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d8102ad0 0x7fa4d819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4d77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4d8102130 0x7fa4d8197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:07.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4dc9a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d8102ad0 0x7fa4d819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48236/0 (socket says 172.21.15.62:48236) 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4dc9a9700 1 -- 172.21.15.62:0/358566708 learned_addr learned my addr 172.21.15.62:0/358566708 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4d77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4d8102130 0x7fa4d8197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.62:57750/0 (socket says 172.21.15.62:57750) 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4d7fff700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d80ff160 0x7fa4d8196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4dc9a9700 1 -- 172.21.15.62:0/358566708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d80ff160 msgr2=0x7fa4d8196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4dc9a9700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d80ff160 0x7fa4d8196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4dc9a9700 1 -- 172.21.15.62:0/358566708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4d8102130 msgr2=0x7fa4d8197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4dc9a9700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4d8102130 0x7fa4d8197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4dc9a9700 1 -- 172.21.15.62:0/358566708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c80095f0 con 0x7fa4d8102ad0 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4d77fe700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4d8102130 0x7fa4d8197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4d7fff700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d80ff160 0x7fa4d8196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.287+0000 7fa4dc9a9700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d8102ad0 0x7fa4d819e380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000ef30 tx=0x7fa4d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.288+0000 7fa4d57fa700 1 -- 172.21.15.62:0/358566708 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0018070 con 0x7fa4d8102ad0 2024-05-08T22:35:07.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.288+0000 7fa4de40c700 1 -- 172.21.15.62:0/358566708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d80fff20 con 0x7fa4d8102ad0 2024-05-08T22:35:07.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.288+0000 7fa4d57fa700 1 -- 172.21.15.62:0/358566708 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4d000f040 con 0x7fa4d8102ad0 2024-05-08T22:35:07.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.288+0000 7fa4d57fa700 1 -- 172.21.15.62:0/358566708 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000cd00 con 0x7fa4d8102ad0 2024-05-08T22:35:07.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.288+0000 7fa4de40c700 1 -- 172.21.15.62:0/358566708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d81003e0 con 0x7fa4d8102ad0 2024-05-08T22:35:07.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.289+0000 7fa4de40c700 1 -- 172.21.15.62:0/358566708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d8108410 con 0x7fa4d8102ad0 2024-05-08T22:35:07.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.290+0000 7fa4d57fa700 1 -- 172.21.15.62:0/358566708 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa4d0003a00 con 0x7fa4d8102ad0 2024-05-08T22:35:07.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.290+0000 7fa4d57fa700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa4c00a1120 0x7fa4c00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:07.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.291+0000 7fa4d7fff700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa4c00a1120 0x7fa4c00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:07.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.291+0000 7fa4d57fa700 1 -- 172.21.15.62:0/358566708 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa4d00cca10 con 0x7fa4d8102ad0 2024-05-08T22:35:07.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.291+0000 7fa4d7fff700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa4c00a1120 0x7fa4c00a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800b680 tx=0x7fa4c8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:07.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.294+0000 7fa4d57fa700 1 -- 172.21.15.62:0/358566708 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa4d0096630 con 0x7fa4d8102ad0 2024-05-08T22:35:07.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:07.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2247991108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:07.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3467179062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:35:07.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:07.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:35:07.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.504+0000 7fa4de40c700 1 -- 172.21.15.62:0/358566708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa4d810c6a0 con 0x7fa4d8102ad0 2024-05-08T22:35:07.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.505+0000 7fa4d57fa700 1 -- 172.21.15.62:0/358566708 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa4d0099c50 con 0x7fa4d8102ad0 2024-05-08T22:35:07.505 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:07.505 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-05-08T22:35:07.505 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:35:07.505 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-05-08T22:35:07.505 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-05-08T22:35:07.505 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:35:07.505 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-05-08T22:35:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 -- 172.21.15.62:0/358566708 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa4c00a1120 msgr2=0x7fa4c00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa4c00a1120 0x7fa4c00a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800b680 tx=0x7fa4c8005fd0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 -- 172.21.15.62:0/358566708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d8102ad0 msgr2=0x7fa4d819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d8102ad0 0x7fa4d819e380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000ef30 tx=0x7fa4d000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 -- 172.21.15.62:0/358566708 shutdown_connections 2024-05-08T22:35:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d8102ad0 0x7fa4d819e380 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4d80ff160 0x7fa4d8196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa4c00a1120 0x7fa4c00a35e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 --2- 172.21.15.62:0/358566708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4d8102130 0x7fa4d8197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 -- 172.21.15.62:0/358566708 >> 172.21.15.62:0/358566708 conn(0x7fa4d80faa90 msgr2=0x7fa4d8105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:07.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 -- 172.21.15.62:0/358566708 shutdown_connections 2024-05-08T22:35:07.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.507+0000 7fa4de40c700 1 -- 172.21.15.62:0/358566708 wait complete. 2024-05-08T22:35:07.519 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T22:35:07.519 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T22:35:07.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:07.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2247991108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:07.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3467179062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:35:07.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:07.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:35:07.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.649+0000 7f0ced1a7700 1 -- 172.21.15.62:0/3866142636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ce8101950 msgr2=0x7f0ce8103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.649+0000 7f0ced1a7700 1 --2- 172.21.15.62:0/3866142636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ce8101950 0x7f0ce8103d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009990 tx=0x7f0cdc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.649+0000 7f0ced1a7700 1 -- 172.21.15.62:0/3866142636 shutdown_connections 2024-05-08T22:35:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.649+0000 7f0ced1a7700 1 --2- 172.21.15.62:0/3866142636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce8106bb0 0x7f0ce8108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.649+0000 7f0ced1a7700 1 --2- 172.21.15.62:0/3866142636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ce8101950 0x7f0ce8103d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.649+0000 7f0ced1a7700 1 --2- 172.21.15.62:0/3866142636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8104280 0x7f0ce8106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.649+0000 7f0ced1a7700 1 -- 172.21.15.62:0/3866142636 >> 172.21.15.62:0/3866142636 conn(0x7f0ce80fb2a0 msgr2=0x7f0ce80fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.649+0000 7f0ced1a7700 1 -- 172.21.15.62:0/3866142636 shutdown_connections 2024-05-08T22:35:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.649+0000 7f0ced1a7700 1 -- 172.21.15.62:0/3866142636 wait complete. 2024-05-08T22:35:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.650+0000 7f0ced1a7700 1 Processor -- start 2024-05-08T22:35:07.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.650+0000 7f0ced1a7700 1 -- start start 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.650+0000 7f0ced1a7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ce8101950 0x7f0ce8196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.650+0000 7f0ced1a7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8104280 0x7f0ce8197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.650+0000 7f0ced1a7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce8106bb0 0x7f0ce819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.650+0000 7f0ced1a7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce819e800 con 0x7f0ce8101950 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ced1a7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce819e970 con 0x7f0ce8104280 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ced1a7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce819eb30 con 0x7f0ce8106bb0 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ce8101950 0x7f0ce8196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ce8101950 0x7f0ce8196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48254/0 (socket says 172.21.15.62:48254) 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce6d9d700 1 -- 172.21.15.62:0/858881541 learned_addr learned my addr 172.21.15.62:0/858881541 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce759e700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce8106bb0 0x7f0ce819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce659c700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8104280 0x7f0ce8197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:07.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce6d9d700 1 -- 172.21.15.62:0/858881541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce8106bb0 msgr2=0x7f0ce819e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce6d9d700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce8106bb0 0x7f0ce819e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce6d9d700 1 -- 172.21.15.62:0/858881541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8104280 msgr2=0x7f0ce8197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce6d9d700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8104280 0x7f0ce8197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce6d9d700 1 -- 172.21.15.62:0/858881541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc0095f0 con 0x7f0ce8101950 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce659c700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8104280 0x7f0ce8197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ce6d9d700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ce8101950 0x7f0ce8196ce0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00b5c0 tx=0x7f0cdc00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ccffff700 1 -- 172.21.15.62:0/858881541 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc01d070 con 0x7f0ce8101950 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ccffff700 1 -- 172.21.15.62:0/858881541 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cdc00fb60 con 0x7f0ce8101950 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ced1a7700 1 -- 172.21.15.62:0/858881541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce819edb0 con 0x7f0ce8101950 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ccffff700 1 -- 172.21.15.62:0/858881541 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc017650 con 0x7f0ce8101950 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.651+0000 7f0ced1a7700 1 -- 172.21.15.62:0/858881541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce819f1d0 con 0x7f0ce8101950 2024-05-08T22:35:07.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.652+0000 7f0ced1a7700 1 -- 172.21.15.62:0/858881541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce80fce20 con 0x7f0ce8101950 2024-05-08T22:35:07.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.653+0000 7f0ccffff700 1 -- 172.21.15.62:0/858881541 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0cdc00fcd0 con 0x7f0ce8101950 2024-05-08T22:35:07.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.654+0000 7f0ccffff700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cd00a1120 0x7f0cd00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:07.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.654+0000 7f0ce659c700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cd00a1120 0x7f0cd00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:07.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.654+0000 7f0ccffff700 1 -- 172.21.15.62:0/858881541 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0cdc0cdbd0 con 0x7f0ce8101950 2024-05-08T22:35:07.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.655+0000 7f0ce659c700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cd00a1120 0x7f0cd00a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8005fd0 tx=0x7f0cd8005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:07.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.658+0000 7f0ccffff700 1 -- 172.21.15.62:0/858881541 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0cdc097890 con 0x7f0ce8101950 2024-05-08T22:35:07.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:07.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2247991108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:07.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3467179062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:35:07.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:35:07.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:35:07.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:07.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:07.866+0000 7f0ced1a7700 1 -- 172.21.15.62:0/858881541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0ce8067480 con 0x7f0ce8101950 2024-05-08T22:35:08.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.368+0000 7f0ccffff700 1 -- 172.21.15.62:0/858881541 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0cdc09aeb0 con 0x7f0ce8101950 2024-05-08T22:35:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 -- 172.21.15.62:0/858881541 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cd00a1120 msgr2=0x7f0cd00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cd00a1120 0x7f0cd00a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8005fd0 tx=0x7f0cd8005e30 comp rx=0 tx=0).stop 2024-05-08T22:35:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 -- 172.21.15.62:0/858881541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ce8101950 msgr2=0x7f0ce8196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ce8101950 0x7f0ce8196ce0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00b5c0 tx=0x7f0cdc00f670 comp rx=0 tx=0).stop 2024-05-08T22:35:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 -- 172.21.15.62:0/858881541 shutdown_connections 2024-05-08T22:35:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce8106bb0 0x7f0ce819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ce8101950 0x7f0ce8196ce0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cd00a1120 0x7f0cd00a35e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 --2- 172.21.15.62:0/858881541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ce8104280 0x7f0ce8197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 -- 172.21.15.62:0/858881541 >> 172.21.15.62:0/858881541 conn(0x7f0ce80fb2a0 msgr2=0x7f0ce8102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 -- 172.21.15.62:0/858881541 shutdown_connections 2024-05-08T22:35:08.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.372+0000 7f0ced1a7700 1 -- 172.21.15.62:0/858881541 wait complete. 2024-05-08T22:35:08.374 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-05-08T22:35:08.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T22:35:08.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.519+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/1293008143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d44102ad0 msgr2=0x7f9d44109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:08.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.519+0000 7f9d4a4c0700 1 --2- 172.21.15.62:0/1293008143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d44102ad0 0x7f9d44109380 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00b0a0 tx=0x7f9d3c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:08.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.519+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/1293008143 shutdown_connections 2024-05-08T22:35:08.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.519+0000 7f9d4a4c0700 1 --2- 172.21.15.62:0/1293008143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d44102ad0 0x7f9d44109380 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:08.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.519+0000 7f9d4a4c0700 1 --2- 172.21.15.62:0/1293008143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d440ff160 0x7f9d440ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:08.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.519+0000 7f9d4a4c0700 1 --2- 172.21.15.62:0/1293008143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d44102130 0x7f9d44102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:08.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.519+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/1293008143 >> 172.21.15.62:0/1293008143 conn(0x7f9d440faa90 msgr2=0x7f9d440fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:08.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.519+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/1293008143 shutdown_connections 2024-05-08T22:35:08.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.519+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/1293008143 wait complete. 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d4a4c0700 1 Processor -- start 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d4a4c0700 1 -- start start 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d4a4c0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d440ff160 0x7f9d44196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d4a4c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d44102130 0x7f9d44197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d4a4c0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d44102ad0 0x7f9d4419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d4a4c0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4419ea00 con 0x7f9d44102ad0 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d4a4c0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4419eb40 con 0x7f9d440ff160 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d4a4c0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4419ecd0 con 0x7f9d44102130 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d48a5d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d44102ad0 0x7f9d4419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d48a5d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d44102ad0 0x7f9d4419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48278/0 (socket says 172.21.15.62:48278) 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d48a5d700 1 -- 172.21.15.62:0/3948809528 learned_addr learned my addr 172.21.15.62:0/3948809528 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.520+0000 7f9d437fe700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d44102130 0x7f9d44197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d43fff700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d440ff160 0x7f9d44196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d48a5d700 1 -- 172.21.15.62:0/3948809528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d44102130 msgr2=0x7f9d44197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d48a5d700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d44102130 0x7f9d44197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d48a5d700 1 -- 172.21.15.62:0/3948809528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d440ff160 msgr2=0x7f9d44196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d48a5d700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d440ff160 0x7f9d44196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d48a5d700 1 -- 172.21.15.62:0/3948809528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c009d00 con 0x7f9d44102ad0 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d43fff700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d440ff160 0x7f9d44196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d437fe700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d44102130 0x7f9d44197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d48a5d700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d44102ad0 0x7f9d4419e430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c004d00 tx=0x7f9d3c004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d417fa700 1 -- 172.21.15.62:0/3948809528 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00e040 con 0x7f9d44102ad0 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d417fa700 1 -- 172.21.15.62:0/3948809528 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d3c01db50 con 0x7f9d44102ad0 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d417fa700 1 -- 172.21.15.62:0/3948809528 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c01bbe0 con 0x7f9d44102ad0 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/3948809528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4419ef50 con 0x7f9d44102ad0 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.521+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/3948809528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4419f310 con 0x7f9d44102ad0 2024-05-08T22:35:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.522+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/3948809528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d44108370 con 0x7f9d44102ad0 2024-05-08T22:35:08.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.523+0000 7f9d417fa700 1 -- 172.21.15.62:0/3948809528 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9d3c01bd40 con 0x7f9d44102ad0 2024-05-08T22:35:08.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.524+0000 7f9d417fa700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d300a1120 0x7f9d300a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:08.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.524+0000 7f9d43fff700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d300a1120 0x7f9d300a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:08.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.524+0000 7f9d417fa700 1 -- 172.21.15.62:0/3948809528 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f9d3c0ce800 con 0x7f9d44102ad0 2024-05-08T22:35:08.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.524+0000 7f9d43fff700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d300a1120 0x7f9d300a35e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9d44069270 tx=0x7f9d34009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:08.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.528+0000 7f9d417fa700 1 -- 172.21.15.62:0/3948809528 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d3c0982b0 con 0x7f9d44102ad0 2024-05-08T22:35:08.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:08 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/358566708' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T22:35:08.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:08 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/858881541' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T22:35:08.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:08 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/358566708' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T22:35:08.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:08 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/858881541' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T22:35:08.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:08.737+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/3948809528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9d44068160 con 0x7f9d44102ad0 2024-05-08T22:35:08.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:08 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/358566708' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T22:35:08.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:08 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/858881541' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T22:35:09.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.374+0000 7f9d417fa700 1 -- 172.21.15.62:0/3948809528 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9d3c09b8d0 con 0x7f9d44102ad0 2024-05-08T22:35:09.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.379+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/3948809528 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d300a1120 msgr2=0x7f9d300a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:09.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.379+0000 7f9d4a4c0700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d300a1120 0x7f9d300a35e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9d44069270 tx=0x7f9d34009670 comp rx=0 tx=0).stop 2024-05-08T22:35:09.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.379+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/3948809528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d44102ad0 msgr2=0x7f9d4419e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:09.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.379+0000 7f9d4a4c0700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d44102ad0 0x7f9d4419e430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c004d00 tx=0x7f9d3c004de0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.385+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/3948809528 shutdown_connections 2024-05-08T22:35:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.385+0000 7f9d4a4c0700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d44102ad0 0x7f9d4419e430 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.385+0000 7f9d4a4c0700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d300a1120 0x7f9d300a35e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.385+0000 7f9d4a4c0700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d440ff160 0x7f9d44196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.385+0000 7f9d4a4c0700 1 --2- 172.21.15.62:0/3948809528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d44102130 0x7f9d44197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.385+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/3948809528 >> 172.21.15.62:0/3948809528 conn(0x7f9d440faa90 msgr2=0x7f9d441054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:09.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.390+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/3948809528 shutdown_connections 2024-05-08T22:35:09.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.392+0000 7f9d4a4c0700 1 -- 172.21.15.62:0/3948809528 wait complete. 2024-05-08T22:35:09.393 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-05-08T22:35:09.406 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T22:35:09.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.535+0000 7f297e006700 1 -- 172.21.15.62:0/3714526849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2978101fd0 msgr2=0x7f2978102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:09.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.535+0000 7f297e006700 1 --2- 172.21.15.62:0/3714526849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2978101fd0 0x7f2978102450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f296c009990 tx=0x7f296c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.535+0000 7f297e006700 1 -- 172.21.15.62:0/3714526849 shutdown_connections 2024-05-08T22:35:09.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.535+0000 7f297e006700 1 --2- 172.21.15.62:0/3714526849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978102990 0x7f29781092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.535+0000 7f297e006700 1 --2- 172.21.15.62:0/3714526849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978100e80 0x7f29781012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.535+0000 7f297e006700 1 --2- 172.21.15.62:0/3714526849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2978101fd0 0x7f2978102450 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.535+0000 7f297e006700 1 -- 172.21.15.62:0/3714526849 >> 172.21.15.62:0/3714526849 conn(0x7f29780fc3f0 msgr2=0x7f29780fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.536+0000 7f297e006700 1 -- 172.21.15.62:0/3714526849 shutdown_connections 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.536+0000 7f297e006700 1 -- 172.21.15.62:0/3714526849 wait complete. 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.536+0000 7f297e006700 1 Processor -- start 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.536+0000 7f297e006700 1 -- start start 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.536+0000 7f297e006700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978100e80 0x7f2978196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f297e006700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2978101fd0 0x7f29781972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f297e006700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978102990 0x7f297819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f297e006700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297819e8d0 con 0x7f2978101fd0 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f297e006700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297819ea10 con 0x7f2978100e80 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f297e006700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297819eba0 con 0x7f2978102990 2024-05-08T22:35:09.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2976ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2978101fd0 0x7f29781972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2976ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2978101fd0 0x7f29781972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48294/0 (socket says 172.21.15.62:48294) 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2976ffd700 1 -- 172.21.15.62:0/3260578649 learned_addr learned my addr 172.21.15.62:0/3260578649 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2977fff700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978102990 0x7f297819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f29777fe700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978100e80 0x7f2978196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2976ffd700 1 -- 172.21.15.62:0/3260578649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978102990 msgr2=0x7f297819e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2976ffd700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978102990 0x7f297819e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2976ffd700 1 -- 172.21.15.62:0/3260578649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978100e80 msgr2=0x7f2978196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2976ffd700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978100e80 0x7f2978196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2976ffd700 1 -- 172.21.15.62:0/3260578649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0095f0 con 0x7f2978101fd0 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f29777fe700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978100e80 0x7f2978196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2976ffd700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2978101fd0 0x7f29781972e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f296c00b5c0 tx=0x7f296c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2974ff9700 1 -- 172.21.15.62:0/3260578649 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c01d070 con 0x7f2978101fd0 2024-05-08T22:35:09.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2974ff9700 1 -- 172.21.15.62:0/3260578649 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f296c004060 con 0x7f2978101fd0 2024-05-08T22:35:09.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f297e006700 1 -- 172.21.15.62:0/3260578649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297819ee20 con 0x7f2978101fd0 2024-05-08T22:35:09.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f2974ff9700 1 -- 172.21.15.62:0/3260578649 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c022e10 con 0x7f2978101fd0 2024-05-08T22:35:09.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.537+0000 7f297e006700 1 -- 172.21.15.62:0/3260578649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297819f1e0 con 0x7f2978101fd0 2024-05-08T22:35:09.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.538+0000 7f297e006700 1 -- 172.21.15.62:0/3260578649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2978108270 con 0x7f2978101fd0 2024-05-08T22:35:09.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.539+0000 7f2974ff9700 1 -- 172.21.15.62:0/3260578649 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f296c022570 con 0x7f2978101fd0 2024-05-08T22:35:09.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.540+0000 7f2974ff9700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f29600a1120 0x7f29600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:09.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.540+0000 7f2974ff9700 1 -- 172.21.15.62:0/3260578649 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f296c0cdf60 con 0x7f2978101fd0 2024-05-08T22:35:09.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.540+0000 7f29777fe700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f29600a1120 0x7f29600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:09.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.541+0000 7f29777fe700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f29600a1120 0x7f29600a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2968005fd0 tx=0x7f2968005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:09.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.544+0000 7f2974ff9700 1 -- 172.21.15.62:0/3260578649 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f296c097920 con 0x7f2978101fd0 2024-05-08T22:35:09.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:09 smithi062 ceph-mon[42571]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:09.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:09 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/858881541' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T22:35:09.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:09 smithi062 ceph-mon[42571]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T22:35:09.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:09 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3948809528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T22:35:09.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:09.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.801+0000 7f297e006700 1 -- 172.21.15.62:0/3260578649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f297810c930 con 0x7f2978101fd0 2024-05-08T22:35:09.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.811+0000 7f2974ff9700 1 -- 172.21.15.62:0/3260578649 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f296c09af40 con 0x7f2978101fd0 2024-05-08T22:35:09.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.815+0000 7f297e006700 1 -- 172.21.15.62:0/3260578649 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f29600a1120 msgr2=0x7f29600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:09.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.815+0000 7f297e006700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f29600a1120 0x7f29600a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2968005fd0 tx=0x7f2968005ee0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.815+0000 7f297e006700 1 -- 172.21.15.62:0/3260578649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2978101fd0 msgr2=0x7f29781972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:09.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.815+0000 7f297e006700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2978101fd0 0x7f29781972e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f296c00b5c0 tx=0x7f296c005e80 comp rx=0 tx=0).stop 2024-05-08T22:35:09.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.818+0000 7f297e006700 1 -- 172.21.15.62:0/3260578649 shutdown_connections 2024-05-08T22:35:09.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.818+0000 7f297e006700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978102990 0x7f297819e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.818+0000 7f297e006700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f29600a1120 0x7f29600a35e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.818+0000 7f297e006700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2978100e80 0x7f2978196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.818+0000 7f297e006700 1 --2- 172.21.15.62:0/3260578649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2978101fd0 0x7f29781972e0 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f296c00b5c0 tx=0x7f296c005e80 comp rx=0 tx=0).stop 2024-05-08T22:35:09.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.818+0000 7f297e006700 1 -- 172.21.15.62:0/3260578649 >> 172.21.15.62:0/3260578649 conn(0x7f29780fc3f0 msgr2=0x7f29781053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:09.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.820+0000 7f297e006700 1 -- 172.21.15.62:0/3260578649 shutdown_connections 2024-05-08T22:35:09.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.820+0000 7f297e006700 1 -- 172.21.15.62:0/3260578649 wait complete. 2024-05-08T22:35:09.822 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T22:35:09.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:09 smithi066 ceph-mon[46830]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:09.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:09 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/858881541' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T22:35:09.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:09 smithi066 ceph-mon[46830]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T22:35:09.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:09 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3948809528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T22:35:09.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:09.841 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:09 smithi079 ceph-mon[46738]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:09 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/858881541' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T22:35:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:09 smithi079 ceph-mon[46738]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T22:35:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:09 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3948809528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T22:35:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:35:09.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.967+0000 7faf17a7f700 1 -- 172.21.15.62:0/931238912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf100ff160 msgr2=0x7faf100ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:09.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.967+0000 7faf17a7f700 1 --2- 172.21.15.62:0/931238912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf100ff160 0x7faf100ff580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009990 tx=0x7faf0c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.967+0000 7faf17a7f700 1 -- 172.21.15.62:0/931238912 shutdown_connections 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.967+0000 7faf17a7f700 1 --2- 172.21.15.62:0/931238912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf10102ad0 0x7faf10109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.967+0000 7faf17a7f700 1 --2- 172.21.15.62:0/931238912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf100ff160 0x7faf100ff580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.967+0000 7faf17a7f700 1 --2- 172.21.15.62:0/931238912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf10102130 0x7faf10102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.967+0000 7faf17a7f700 1 -- 172.21.15.62:0/931238912 >> 172.21.15.62:0/931238912 conn(0x7faf100faa90 msgr2=0x7faf100fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.967+0000 7faf17a7f700 1 -- 172.21.15.62:0/931238912 shutdown_connections 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf17a7f700 1 -- 172.21.15.62:0/931238912 wait complete. 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf17a7f700 1 Processor -- start 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf17a7f700 1 -- start start 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf17a7f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf100ff160 0x7faf10101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf17a7f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf10102130 0x7faf101000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf17a7f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf10102ad0 0x7faf10100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf17a7f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf10100dc0 con 0x7faf10102ad0 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf17a7f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf10100f30 con 0x7faf100ff160 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf17a7f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf101a0f20 con 0x7faf10102130 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf1601c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf10102ad0 0x7faf10100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf1581b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf100ff160 0x7faf10101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf1601c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf10102ad0 0x7faf10100620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48316/0 (socket says 172.21.15.62:48316) 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf1581b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf100ff160 0x7faf10101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.62:57836/0 (socket says 172.21.15.62:57836) 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf1601c700 1 -- 172.21.15.62:0/3406465209 learned_addr learned my addr 172.21.15.62:0/3406465209 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.968+0000 7faf1501a700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf10102130 0x7faf101000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.969+0000 7faf1601c700 1 -- 172.21.15.62:0/3406465209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf10102130 msgr2=0x7faf101000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.969+0000 7faf1601c700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf10102130 0x7faf101000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.969+0000 7faf1601c700 1 -- 172.21.15.62:0/3406465209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf100ff160 msgr2=0x7faf10101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.969+0000 7faf1601c700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf100ff160 0x7faf10101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.969+0000 7faf1601c700 1 -- 172.21.15.62:0/3406465209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0095f0 con 0x7faf10102ad0 2024-05-08T22:35:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.969+0000 7faf1601c700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf10102ad0 0x7faf10100620 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf08007ee0 tx=0x7faf0800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.969+0000 7faf02ffd700 1 -- 172.21.15.62:0/3406465209 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800dcc0 con 0x7faf10102ad0 2024-05-08T22:35:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.969+0000 7faf02ffd700 1 -- 172.21.15.62:0/3406465209 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf0800f040 con 0x7faf10102ad0 2024-05-08T22:35:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.969+0000 7faf17a7f700 1 -- 172.21.15.62:0/3406465209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf101a11b0 con 0x7faf10102ad0 2024-05-08T22:35:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.969+0000 7faf02ffd700 1 -- 172.21.15.62:0/3406465209 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08004830 con 0x7faf10102ad0 2024-05-08T22:35:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.969+0000 7faf17a7f700 1 -- 172.21.15.62:0/3406465209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf101a1670 con 0x7faf10102ad0 2024-05-08T22:35:09.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.970+0000 7faf17a7f700 1 -- 172.21.15.62:0/3406465209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf10108460 con 0x7faf10102ad0 2024-05-08T22:35:09.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.971+0000 7faf02ffd700 1 -- 172.21.15.62:0/3406465209 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7faf08003a00 con 0x7faf10102ad0 2024-05-08T22:35:09.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.972+0000 7faf02ffd700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faefc0a1120 0x7faefc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:09.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.972+0000 7faf02ffd700 1 -- 172.21.15.62:0/3406465209 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faf080cc860 con 0x7faf10102ad0 2024-05-08T22:35:09.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.972+0000 7faf1581b700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faefc0a1120 0x7faefc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:09.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.972+0000 7faf1581b700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faefc0a1120 0x7faefc0a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b5c0 tx=0x7faf0c003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:09.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:09.975+0000 7faf02ffd700 1 -- 172.21.15.62:0/3406465209 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faf08096140 con 0x7faf10102ad0 2024-05-08T22:35:10.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.184+0000 7faf17a7f700 1 -- 172.21.15.62:0/3406465209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf1010c720 con 0x7faf10102ad0 2024-05-08T22:35:10.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.186+0000 7faf02ffd700 1 -- 172.21.15.62:0/3406465209 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+22060 (secure 0 0 0) 0x7faf08023090 con 0x7faf10102ad0 2024-05-08T22:35:10.186 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:10.186 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:35:09.802988+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:10.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 -- 172.21.15.62:0/3406465209 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faefc0a1120 msgr2=0x7faefc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:10.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faefc0a1120 0x7faefc0a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b5c0 tx=0x7faf0c003790 comp rx=0 tx=0).stop 2024-05-08T22:35:10.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 -- 172.21.15.62:0/3406465209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf10102ad0 msgr2=0x7faf10100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:10.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf10102ad0 0x7faf10100620 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf08007ee0 tx=0x7faf0800d3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 -- 172.21.15.62:0/3406465209 shutdown_connections 2024-05-08T22:35:10.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf10102ad0 0x7faf10100620 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faefc0a1120 0x7faefc0a35e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf100ff160 0x7faf10101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 --2- 172.21.15.62:0/3406465209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf10102130 0x7faf101000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 -- 172.21.15.62:0/3406465209 >> 172.21.15.62:0/3406465209 conn(0x7faf100faa90 msgr2=0x7faf101055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:10.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 -- 172.21.15.62:0/3406465209 shutdown_connections 2024-05-08T22:35:10.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.188+0000 7faf17a7f700 1 -- 172.21.15.62:0/3406465209 wait complete. 2024-05-08T22:35:10.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:10.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.331+0000 7f713a456700 1 -- 172.21.15.62:0/2131823017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7134100f70 msgr2=0x7f71341013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:10.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.331+0000 7f713a456700 1 --2- 172.21.15.62:0/2131823017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7134100f70 0x7f71341013d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71280099c0 tx=0x7f7128009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.331+0000 7f713a456700 1 -- 172.21.15.62:0/2131823017 shutdown_connections 2024-05-08T22:35:10.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.331+0000 7f713a456700 1 --2- 172.21.15.62:0/2131823017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7134101910 0x7f71341081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.331+0000 7f713a456700 1 --2- 172.21.15.62:0/2131823017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71340ffd80 0x7f71341001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.331+0000 7f713a456700 1 --2- 172.21.15.62:0/2131823017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7134100f70 0x7f71341013d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.331+0000 7f713a456700 1 -- 172.21.15.62:0/2131823017 >> 172.21.15.62:0/2131823017 conn(0x7f71340fb310 msgr2=0x7f71340fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.331+0000 7f713a456700 1 -- 172.21.15.62:0/2131823017 shutdown_connections 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.331+0000 7f713a456700 1 -- 172.21.15.62:0/2131823017 wait complete. 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.332+0000 7f713a456700 1 Processor -- start 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.332+0000 7f713a456700 1 -- start start 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.332+0000 7f713a456700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340ffd80 0x7f7134195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.332+0000 7f713a456700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7134100f70 0x7f71341962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.332+0000 7f713a456700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7134101910 0x7f713419d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.332+0000 7f713a456700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713419d870 con 0x7f71340ffd80 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.332+0000 7f713a456700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713419d9e0 con 0x7f7134101910 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.332+0000 7f713a456700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713419dba0 con 0x7f7134100f70 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.332+0000 7f7133fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340ffd80 0x7f7134195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:10.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.332+0000 7f7133fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340ffd80 0x7f7134195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48340/0 (socket says 172.21.15.62:48340) 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f7133fff700 1 -- 172.21.15.62:0/2768091319 learned_addr learned my addr 172.21.15.62:0/2768091319 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f71337fe700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7134100f70 0x7f71341962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f71389f3700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7134101910 0x7f713419d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f7133fff700 1 -- 172.21.15.62:0/2768091319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7134100f70 msgr2=0x7f71341962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f7133fff700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7134100f70 0x7f71341962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f7133fff700 1 -- 172.21.15.62:0/2768091319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7134101910 msgr2=0x7f713419d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f7133fff700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7134101910 0x7f713419d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f7133fff700 1 -- 172.21.15.62:0/2768091319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71280095f0 con 0x7f71340ffd80 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f7133fff700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340ffd80 0x7f7134195d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f712400b500 tx=0x7f712400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f71317fa700 1 -- 172.21.15.62:0/2768091319 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124011830 con 0x7f71340ffd80 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f71317fa700 1 -- 172.21.15.62:0/2768091319 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7124011e50 con 0x7f71340ffd80 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f713a456700 1 -- 172.21.15.62:0/2768091319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713419de80 con 0x7f71340ffd80 2024-05-08T22:35:10.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f71317fa700 1 -- 172.21.15.62:0/2768091319 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124010dc0 con 0x7f71340ffd80 2024-05-08T22:35:10.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.333+0000 7f713a456700 1 -- 172.21.15.62:0/2768091319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713419e2b0 con 0x7f71340ffd80 2024-05-08T22:35:10.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.334+0000 7f713a456700 1 -- 172.21.15.62:0/2768091319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7134107130 con 0x7f71340ffd80 2024-05-08T22:35:10.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.335+0000 7f71317fa700 1 -- 172.21.15.62:0/2768091319 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7124010380 con 0x7f71340ffd80 2024-05-08T22:35:10.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.336+0000 7f71317fa700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f711c0a11a0 0x7f711c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:10.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.336+0000 7f71337fe700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f711c0a11a0 0x7f711c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:10.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.336+0000 7f71317fa700 1 -- 172.21.15.62:0/2768091319 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f71240cc6b0 con 0x7f71340ffd80 2024-05-08T22:35:10.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.336+0000 7f71337fe700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f711c0a11a0 0x7f711c0a3660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7128006010 tx=0x7f7128005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:10.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.340+0000 7f71317fa700 1 -- 172.21.15.62:0/2768091319 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7124095f90 con 0x7f71340ffd80 2024-05-08T22:35:10.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.549+0000 7f713a456700 1 -- 172.21.15.62:0/2768091319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7134067d10 con 0x7f71340ffd80 2024-05-08T22:35:10.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.550+0000 7f71317fa700 1 -- 172.21.15.62:0/2768091319 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+22060 (secure 0 0 0) 0x7f7124015090 con 0x7f71340ffd80 2024-05-08T22:35:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:10.551 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:35:09.802988+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:10.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.552+0000 7f713a456700 1 -- 172.21.15.62:0/2768091319 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f711c0a11a0 msgr2=0x7f711c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:10.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.552+0000 7f713a456700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f711c0a11a0 0x7f711c0a3660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7128006010 tx=0x7f7128005bc0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.552+0000 7f713a456700 1 -- 172.21.15.62:0/2768091319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340ffd80 msgr2=0x7f7134195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:10.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.552+0000 7f713a456700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340ffd80 0x7f7134195d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f712400b500 tx=0x7f712400b810 comp rx=0 tx=0).stop 2024-05-08T22:35:10.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.553+0000 7f713a456700 1 -- 172.21.15.62:0/2768091319 shutdown_connections 2024-05-08T22:35:10.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.553+0000 7f713a456700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f711c0a11a0 0x7f711c0a3660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.553+0000 7f713a456700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7134101910 0x7f713419d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.553+0000 7f713a456700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340ffd80 0x7f7134195d80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.553+0000 7f713a456700 1 --2- 172.21.15.62:0/2768091319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7134100f70 0x7f71341962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.553+0000 7f713a456700 1 -- 172.21.15.62:0/2768091319 >> 172.21.15.62:0/2768091319 conn(0x7f71340fb310 msgr2=0x7f7134104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:10.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.553+0000 7f713a456700 1 -- 172.21.15.62:0/2768091319 shutdown_connections 2024-05-08T22:35:10.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.553+0000 7f713a456700 1 -- 172.21.15.62:0/2768091319 wait complete. 2024-05-08T22:35:10.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:10 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3948809528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T22:35:10.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:10 smithi062 ceph-mon[42571]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T22:35:10.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:10 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3260578649' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T22:35:10.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:10 smithi062 ceph-mon[42571]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T22:35:10.565 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:10 smithi062 ceph-mon[42571]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T22:35:10.565 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:10 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3260578649' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T22:35:10.565 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:10 smithi062 ceph-mon[42571]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T22:35:10.565 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:10 smithi062 ceph-mon[42571]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T22:35:10.565 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:10 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3406465209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:10.565 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:10 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] up:active 2024-05-08T22:35:10.565 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:10 smithi062 ceph-mon[42571]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:35:10.567 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T22:35:10.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:10 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3948809528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T22:35:10.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:10 smithi079 ceph-mon[46738]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T22:35:10.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:10 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3260578649' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T22:35:10.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:10 smithi079 ceph-mon[46738]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T22:35:10.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:10 smithi079 ceph-mon[46738]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T22:35:10.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:10 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3260578649' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T22:35:10.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:10 smithi079 ceph-mon[46738]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T22:35:10.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:10 smithi079 ceph-mon[46738]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T22:35:10.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:10 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3406465209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:10.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:10 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] up:active 2024-05-08T22:35:10.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:10 smithi079 ceph-mon[46738]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.696+0000 7f39435b8700 1 -- 172.21.15.62:0/913995931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f393c1018f0 msgr2=0x7f393c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.696+0000 7f39435b8700 1 --2- 172.21.15.62:0/913995931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f393c1018f0 0x7f393c1081a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f393400b0d0 tx=0x7f393400b3e0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.696+0000 7f39435b8700 1 -- 172.21.15.62:0/913995931 shutdown_connections 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.696+0000 7f39435b8700 1 --2- 172.21.15.62:0/913995931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f393c1018f0 0x7f393c1081a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.696+0000 7f39435b8700 1 --2- 172.21.15.62:0/913995931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f393c0ffd60 0x7f393c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.696+0000 7f39435b8700 1 --2- 172.21.15.62:0/913995931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c100f50 0x7f393c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.696+0000 7f39435b8700 1 -- 172.21.15.62:0/913995931 >> 172.21.15.62:0/913995931 conn(0x7f393c0fb290 msgr2=0x7f393c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.696+0000 7f39435b8700 1 -- 172.21.15.62:0/913995931 shutdown_connections 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.696+0000 7f39435b8700 1 -- 172.21.15.62:0/913995931 wait complete. 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f39435b8700 1 Processor -- start 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f39435b8700 1 -- start start 2024-05-08T22:35:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f39435b8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c0ffd60 0x7f393c195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f39435b8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f393c100f50 0x7f393c196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f39435b8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f393c1018f0 0x7f393c19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f39435b8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c19d840 con 0x7f393c100f50 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f39435b8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c19d9b0 con 0x7f393c0ffd60 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f39435b8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c19db70 con 0x7f393c1018f0 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3940b53700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f393c100f50 0x7f393c196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3940b53700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f393c100f50 0x7f393c196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48358/0 (socket says 172.21.15.62:48358) 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3940b53700 1 -- 172.21.15.62:0/2373689207 learned_addr learned my addr 172.21.15.62:0/2373689207 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3941354700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c0ffd60 0x7f393c195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3941b55700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f393c1018f0 0x7f393c19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3940b53700 1 -- 172.21.15.62:0/2373689207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f393c1018f0 msgr2=0x7f393c19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3940b53700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f393c1018f0 0x7f393c19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3940b53700 1 -- 172.21.15.62:0/2373689207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c0ffd60 msgr2=0x7f393c195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3940b53700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c0ffd60 0x7f393c195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:10.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3940b53700 1 -- 172.21.15.62:0/2373689207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3934009d00 con 0x7f393c100f50 2024-05-08T22:35:10.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3941b55700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f393c1018f0 0x7f393c19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:10.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.697+0000 7f3940b53700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f393c100f50 0x7f393c196290 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f393800c9b0 tx=0x7f393800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:10.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.698+0000 7f39327fc700 1 -- 172.21.15.62:0/2373689207 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938007b00 con 0x7f393c100f50 2024-05-08T22:35:10.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.698+0000 7f39435b8700 1 -- 172.21.15.62:0/2373689207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c19de50 con 0x7f393c100f50 2024-05-08T22:35:10.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.698+0000 7f39327fc700 1 -- 172.21.15.62:0/2373689207 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3938007c60 con 0x7f393c100f50 2024-05-08T22:35:10.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.698+0000 7f39327fc700 1 -- 172.21.15.62:0/2373689207 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938018660 con 0x7f393c100f50 2024-05-08T22:35:10.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.698+0000 7f39435b8700 1 -- 172.21.15.62:0/2373689207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c19e2d0 con 0x7f393c100f50 2024-05-08T22:35:10.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.699+0000 7f39435b8700 1 -- 172.21.15.62:0/2373689207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393c1070b0 con 0x7f393c100f50 2024-05-08T22:35:10.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.700+0000 7f39327fc700 1 -- 172.21.15.62:0/2373689207 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f393800f450 con 0x7f393c100f50 2024-05-08T22:35:10.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.700+0000 7f39327fc700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39280a1120 0x7f39280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:10.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.700+0000 7f3941354700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39280a1120 0x7f39280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:10.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.701+0000 7f39327fc700 1 -- 172.21.15.62:0/2373689207 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f39380cca80 con 0x7f393c100f50 2024-05-08T22:35:10.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.701+0000 7f3941354700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39280a1120 0x7f39280a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f393c1971d0 tx=0x7f392c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.704+0000 7f39327fc700 1 -- 172.21.15.62:0/2373689207 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f39380963f0 con 0x7f393c100f50 2024-05-08T22:35:10.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:10 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3948809528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T22:35:10.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:10 smithi066 ceph-mon[46830]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T22:35:10.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:10 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3260578649' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T22:35:10.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:10 smithi066 ceph-mon[46830]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T22:35:10.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:10 smithi066 ceph-mon[46830]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T22:35:10.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:10 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3260578649' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T22:35:10.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:10 smithi066 ceph-mon[46830]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T22:35:10.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:10 smithi066 ceph-mon[46830]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T22:35:10.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:10 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3406465209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:10.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:10 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] up:active 2024-05-08T22:35:10.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:10 smithi066 ceph-mon[46830]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:35:10.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:10.964+0000 7f39435b8700 1 -- 172.21.15.62:0/2373689207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f393c067d10 con 0x7f393c100f50 2024-05-08T22:35:11.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:11 smithi079 ceph-mon[46738]: pgmap v221: 97 pgs: 19 creating+peering, 10 active+clean, 68 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:11.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:11 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2768091319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:11.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:11 smithi079 ceph-mon[46738]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T22:35:11.639 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:11 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2373689207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T22:35:11.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:11 smithi062 ceph-mon[42571]: pgmap v221: 97 pgs: 19 creating+peering, 10 active+clean, 68 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:11.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:11 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2768091319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:11.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:11 smithi062 ceph-mon[42571]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T22:35:11.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:11 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2373689207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T22:35:11.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.813+0000 7f39327fc700 1 -- 172.21.15.62:0/2373689207 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3938099a10 con 0x7f393c100f50 2024-05-08T22:35:11.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 -- 172.21.15.62:0/2373689207 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39280a1120 msgr2=0x7f39280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39280a1120 0x7f39280a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f393c1971d0 tx=0x7f392c00a380 comp rx=0 tx=0).stop 2024-05-08T22:35:11.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 -- 172.21.15.62:0/2373689207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f393c100f50 msgr2=0x7f393c196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f393c100f50 0x7f393c196290 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f393800c9b0 tx=0x7f393800ccc0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 -- 172.21.15.62:0/2373689207 shutdown_connections 2024-05-08T22:35:11.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f393c1018f0 0x7f393c19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39280a1120 0x7f39280a35e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c0ffd60 0x7f393c195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 --2- 172.21.15.62:0/2373689207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f393c100f50 0x7f393c196290 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 -- 172.21.15.62:0/2373689207 >> 172.21.15.62:0/2373689207 conn(0x7f393c0fb290 msgr2=0x7f393c1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 -- 172.21.15.62:0/2373689207 shutdown_connections 2024-05-08T22:35:11.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.816+0000 7f39435b8700 1 -- 172.21.15.62:0/2373689207 wait complete. 2024-05-08T22:35:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:11 smithi066 ceph-mon[46830]: pgmap v221: 97 pgs: 19 creating+peering, 10 active+clean, 68 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T22:35:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:11 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2768091319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:11 smithi066 ceph-mon[46830]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T22:35:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:11 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2373689207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T22:35:11.831 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-08T22:35:11.832 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-05-08T22:35:11.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.969+0000 7f9d77d52700 1 -- 172.21.15.62:0/2541746573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d70100f50 msgr2=0x7f9d701013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.969+0000 7f9d77d52700 1 --2- 172.21.15.62:0/2541746573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d70100f50 0x7f9d701013b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0099c0 tx=0x7f9d6c009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.969+0000 7f9d77d52700 1 -- 172.21.15.62:0/2541746573 shutdown_connections 2024-05-08T22:35:11.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.969+0000 7f9d77d52700 1 --2- 172.21.15.62:0/2541746573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d701018f0 0x7f9d701081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.969+0000 7f9d77d52700 1 --2- 172.21.15.62:0/2541746573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d700ffd60 0x7f9d70100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.969+0000 7f9d77d52700 1 --2- 172.21.15.62:0/2541746573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d70100f50 0x7f9d701013b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.969+0000 7f9d77d52700 1 -- 172.21.15.62:0/2541746573 >> 172.21.15.62:0/2541746573 conn(0x7f9d700fb290 msgr2=0x7f9d700fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:11.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.969+0000 7f9d77d52700 1 -- 172.21.15.62:0/2541746573 shutdown_connections 2024-05-08T22:35:11.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.969+0000 7f9d77d52700 1 -- 172.21.15.62:0/2541746573 wait complete. 2024-05-08T22:35:11.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d77d52700 1 Processor -- start 2024-05-08T22:35:11.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d77d52700 1 -- start start 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d77d52700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d700ffd60 0x7f9d70196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d77d52700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70100f50 0x7f9d70197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d77d52700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d701018f0 0x7f9d7019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d77d52700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7019ea00 con 0x7f9d701018f0 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d77d52700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7019eb40 con 0x7f9d70100f50 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d77d52700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7019ecd0 con 0x7f9d700ffd60 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d762ef700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d701018f0 0x7f9d7019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d762ef700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d701018f0 0x7f9d7019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48366/0 (socket says 172.21.15.62:48366) 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d762ef700 1 -- 172.21.15.62:0/2019814224 learned_addr learned my addr 172.21.15.62:0/2019814224 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d75aee700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d700ffd60 0x7f9d70196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.970+0000 7f9d752ed700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70100f50 0x7f9d70197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.971+0000 7f9d762ef700 1 -- 172.21.15.62:0/2019814224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d700ffd60 msgr2=0x7f9d70196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.971+0000 7f9d762ef700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d700ffd60 0x7f9d70196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.971+0000 7f9d762ef700 1 -- 172.21.15.62:0/2019814224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70100f50 msgr2=0x7f9d70197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.971+0000 7f9d762ef700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70100f50 0x7f9d70197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.971+0000 7f9d762ef700 1 -- 172.21.15.62:0/2019814224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0095f0 con 0x7f9d701018f0 2024-05-08T22:35:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.971+0000 7f9d762ef700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d701018f0 0x7f9d7019e430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9d68007ee0 tx=0x7f9d6800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.971+0000 7f9d66ffd700 1 -- 172.21.15.62:0/2019814224 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68017070 con 0x7f9d701018f0 2024-05-08T22:35:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.971+0000 7f9d66ffd700 1 -- 172.21.15.62:0/2019814224 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d6800f040 con 0x7f9d701018f0 2024-05-08T22:35:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.971+0000 7f9d77d52700 1 -- 172.21.15.62:0/2019814224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7019efb0 con 0x7f9d701018f0 2024-05-08T22:35:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.971+0000 7f9d66ffd700 1 -- 172.21.15.62:0/2019814224 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68005580 con 0x7f9d701018f0 2024-05-08T22:35:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.971+0000 7f9d77d52700 1 -- 172.21.15.62:0/2019814224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7019f3e0 con 0x7f9d701018f0 2024-05-08T22:35:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.972+0000 7f9d77d52700 1 -- 172.21.15.62:0/2019814224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d70067d10 con 0x7f9d701018f0 2024-05-08T22:35:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.973+0000 7f9d66ffd700 1 -- 172.21.15.62:0/2019814224 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9d68022020 con 0x7f9d701018f0 2024-05-08T22:35:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.974+0000 7f9d66ffd700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a1120 0x7f9d5c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.974+0000 7f9d75aee700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a1120 0x7f9d5c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.974+0000 7f9d66ffd700 1 -- 172.21.15.62:0/2019814224 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9d680cbd90 con 0x7f9d701018f0 2024-05-08T22:35:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.974+0000 7f9d75aee700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a1120 0x7f9d5c0a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d70100db0 tx=0x7f9d60009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:11.977+0000 7f9d66ffd700 1 -- 172.21.15.62:0/2019814224 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d68095670 con 0x7f9d701018f0 2024-05-08T22:35:12.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.233+0000 7f9d77d52700 1 -- 172.21.15.62:0/2019814224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f9d7010b8b0 con 0x7f9d701018f0 2024-05-08T22:35:12.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.823+0000 7f9d66ffd700 1 -- 172.21.15.62:0/2019814224 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) v1 ==== 105+0+0 (secure 0 0 0) 0x7f9d68098c90 con 0x7f9d701018f0 2024-05-08T22:35:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.826+0000 7f9d77d52700 1 -- 172.21.15.62:0/2019814224 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a1120 msgr2=0x7f9d5c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.826+0000 7f9d77d52700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a1120 0x7f9d5c0a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d70100db0 tx=0x7f9d60009190 comp rx=0 tx=0).stop 2024-05-08T22:35:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.826+0000 7f9d77d52700 1 -- 172.21.15.62:0/2019814224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d701018f0 msgr2=0x7f9d7019e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.826+0000 7f9d77d52700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d701018f0 0x7f9d7019e430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9d68007ee0 tx=0x7f9d6800d3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.826+0000 7f9d77d52700 1 -- 172.21.15.62:0/2019814224 shutdown_connections 2024-05-08T22:35:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.826+0000 7f9d77d52700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d701018f0 0x7f9d7019e430 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.826+0000 7f9d77d52700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d700ffd60 0x7f9d70196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.826+0000 7f9d77d52700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a1120 0x7f9d5c0a35e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.826+0000 7f9d77d52700 1 --2- 172.21.15.62:0/2019814224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d70100f50 0x7f9d70197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.826+0000 7f9d77d52700 1 -- 172.21.15.62:0/2019814224 >> 172.21.15.62:0/2019814224 conn(0x7f9d700fb290 msgr2=0x7f9d700fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:12.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.829+0000 7f9d77d52700 1 -- 172.21.15.62:0/2019814224 shutdown_connections 2024-05-08T22:35:12.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.829+0000 7f9d77d52700 1 -- 172.21.15.62:0/2019814224 wait complete. 2024-05-08T22:35:12.846 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-08T22:35:12.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.982+0000 7f0060437700 1 -- 172.21.15.62:0/2606529760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0058101fc0 msgr2=0x7f0058102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.982+0000 7f0060437700 1 --2- 172.21.15.62:0/2606529760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0058101fc0 0x7f0058102440 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0054009990 tx=0x7f0054009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.982+0000 7f0060437700 1 -- 172.21.15.62:0/2606529760 shutdown_connections 2024-05-08T22:35:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.982+0000 7f0060437700 1 --2- 172.21.15.62:0/2606529760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0058102980 0x7f00581092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.982+0000 7f0060437700 1 --2- 172.21.15.62:0/2606529760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0058100e70 0x7f0058101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.982+0000 7f0060437700 1 --2- 172.21.15.62:0/2606529760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0058101fc0 0x7f0058102440 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.982+0000 7f0060437700 1 -- 172.21.15.62:0/2606529760 >> 172.21.15.62:0/2606529760 conn(0x7f00580fc400 msgr2=0x7f00580fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.982+0000 7f0060437700 1 -- 172.21.15.62:0/2606529760 shutdown_connections 2024-05-08T22:35:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.982+0000 7f0060437700 1 -- 172.21.15.62:0/2606529760 wait complete. 2024-05-08T22:35:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f0060437700 1 Processor -- start 2024-05-08T22:35:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f0060437700 1 -- start start 2024-05-08T22:35:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f0060437700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0058100e70 0x7f0058074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f0060437700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0058101fc0 0x7f0058073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f0060437700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0058102980 0x7f00580738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f0060437700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0058074070 con 0x7f0058102980 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f0060437700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00580741e0 con 0x7f0058100e70 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f0060437700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00581a0de0 con 0x7f0058101fc0 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f005e9d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0058102980 0x7f00580738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f005e9d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0058102980 0x7f00580738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48384/0 (socket says 172.21.15.62:48384) 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f005e9d4700 1 -- 172.21.15.62:0/1823056131 learned_addr learned my addr 172.21.15.62:0/1823056131 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f005d9d2700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0058101fc0 0x7f0058073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f005e1d3700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0058100e70 0x7f0058074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f005e9d4700 1 -- 172.21.15.62:0/1823056131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0058101fc0 msgr2=0x7f0058073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f005e9d4700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0058101fc0 0x7f0058073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f005e9d4700 1 -- 172.21.15.62:0/1823056131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0058100e70 msgr2=0x7f0058074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f005e9d4700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0058100e70 0x7f0058074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.983+0000 7f005e9d4700 1 -- 172.21.15.62:0/1823056131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00540095f0 con 0x7f0058102980 2024-05-08T22:35:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.984+0000 7f005e9d4700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0058102980 0x7f00580738d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0050007d70 tx=0x7f005000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.984+0000 7f004f7fe700 1 -- 172.21.15.62:0/1823056131 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005000ac90 con 0x7f0058102980 2024-05-08T22:35:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.984+0000 7f004f7fe700 1 -- 172.21.15.62:0/1823056131 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0050007e60 con 0x7f0058102980 2024-05-08T22:35:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.984+0000 7f0060437700 1 -- 172.21.15.62:0/1823056131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00581a1070 con 0x7f0058102980 2024-05-08T22:35:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.984+0000 7f004f7fe700 1 -- 172.21.15.62:0/1823056131 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00500115e0 con 0x7f0058102980 2024-05-08T22:35:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.984+0000 7f0060437700 1 -- 172.21.15.62:0/1823056131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00581a1530 con 0x7f0058102980 2024-05-08T22:35:12.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.985+0000 7f0060437700 1 -- 172.21.15.62:0/1823056131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0058108290 con 0x7f0058102980 2024-05-08T22:35:12.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.986+0000 7f004f7fe700 1 -- 172.21.15.62:0/1823056131 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0050012420 con 0x7f0058102980 2024-05-08T22:35:12.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.987+0000 7f004f7fe700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f00440a1120 0x7f00440a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:12.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.987+0000 7f005e1d3700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f00440a1120 0x7f00440a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:12.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.987+0000 7f004f7fe700 1 -- 172.21.15.62:0/1823056131 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f00500ccef0 con 0x7f0058102980 2024-05-08T22:35:12.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.987+0000 7f005e1d3700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f00440a1120 0x7f00440a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0048005950 tx=0x7f004800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:12.990+0000 7f004f7fe700 1 -- 172.21.15.62:0/1823056131 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0050096730 con 0x7f0058102980 2024-05-08T22:35:13.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:12 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2373689207' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T22:35:13.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:12 smithi066 ceph-mon[46830]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:35:13.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:12 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2019814224' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T22:35:13.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:12 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2373689207' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T22:35:13.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:12 smithi079 ceph-mon[46738]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:35:13.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:12 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2019814224' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T22:35:13.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:12 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2373689207' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T22:35:13.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:12 smithi062 ceph-mon[42571]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:35:13.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:12 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2019814224' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T22:35:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.247+0000 7f0060437700 1 -- 172.21.15.62:0/1823056131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f0058067480 con 0x7f0058102980 2024-05-08T22:35:13.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.833+0000 7f004f7fe700 1 -- 172.21.15.62:0/1823056131 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f0050099d50 con 0x7f0058102980 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.835+0000 7f0060437700 1 -- 172.21.15.62:0/1823056131 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f00440a1120 msgr2=0x7f00440a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.835+0000 7f0060437700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f00440a1120 0x7f00440a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0048005950 tx=0x7f004800b410 comp rx=0 tx=0).stop 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.835+0000 7f0060437700 1 -- 172.21.15.62:0/1823056131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0058102980 msgr2=0x7f00580738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.835+0000 7f0060437700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0058102980 0x7f00580738d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0050007d70 tx=0x7f005000a340 comp rx=0 tx=0).stop 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.836+0000 7f0060437700 1 -- 172.21.15.62:0/1823056131 shutdown_connections 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.836+0000 7f0060437700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0058102980 0x7f00580738d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.836+0000 7f0060437700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f00440a1120 0x7f00440a35e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.836+0000 7f0060437700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0058100e70 0x7f0058074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.836+0000 7f0060437700 1 --2- 172.21.15.62:0/1823056131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0058101fc0 0x7f0058073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.836+0000 7f0060437700 1 -- 172.21.15.62:0/1823056131 >> 172.21.15.62:0/1823056131 conn(0x7f00580fc400 msgr2=0x7f00581053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.836+0000 7f0060437700 1 -- 172.21.15.62:0/1823056131 shutdown_connections 2024-05-08T22:35:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.836+0000 7f0060437700 1 -- 172.21.15.62:0/1823056131 wait complete. 2024-05-08T22:35:13.853 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-08T22:35:13.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.986+0000 7f96b03a3700 1 -- 172.21.15.62:0/1661755090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a80ffd60 msgr2=0x7f96a8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.986+0000 7f96b03a3700 1 --2- 172.21.15.62:0/1661755090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a80ffd60 0x7f96a8100180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96980099c0 tx=0x7f9698009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.986+0000 7f96b03a3700 1 -- 172.21.15.62:0/1661755090 shutdown_connections 2024-05-08T22:35:13.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.986+0000 7f96b03a3700 1 --2- 172.21.15.62:0/1661755090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96a81018f0 0x7f96a81081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.986+0000 7f96b03a3700 1 --2- 172.21.15.62:0/1661755090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a80ffd60 0x7f96a8100180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.986+0000 7f96b03a3700 1 --2- 172.21.15.62:0/1661755090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a8100f50 0x7f96a81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.986+0000 7f96b03a3700 1 -- 172.21.15.62:0/1661755090 >> 172.21.15.62:0/1661755090 conn(0x7f96a80fb290 msgr2=0x7f96a80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:13.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.986+0000 7f96b03a3700 1 -- 172.21.15.62:0/1661755090 shutdown_connections 2024-05-08T22:35:13.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.986+0000 7f96b03a3700 1 -- 172.21.15.62:0/1661755090 wait complete. 2024-05-08T22:35:13.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96b03a3700 1 Processor -- start 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96b03a3700 1 -- start start 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96b03a3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a80ffd60 0x7f96a8068be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96b03a3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96a8100f50 0x7f96a819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96b03a3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a81018f0 0x7f96a819e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96b03a3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a819e8b0 con 0x7f96a81018f0 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96b03a3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a819ea20 con 0x7f96a80ffd60 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96b03a3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a819ebe0 con 0x7f96a8100f50 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96ae940700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a81018f0 0x7f96a819e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96ae940700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a81018f0 0x7f96a819e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48404/0 (socket says 172.21.15.62:48404) 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96ae940700 1 -- 172.21.15.62:0/3982737371 learned_addr learned my addr 172.21.15.62:0/3982737371 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96ae13f700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a80ffd60 0x7f96a8068be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96ad93e700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96a8100f50 0x7f96a819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96ae940700 1 -- 172.21.15.62:0/3982737371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96a8100f50 msgr2=0x7f96a819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96ae940700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96a8100f50 0x7f96a819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96ae940700 1 -- 172.21.15.62:0/3982737371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a80ffd60 msgr2=0x7f96a8068be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96ae940700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a80ffd60 0x7f96a8068be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.987+0000 7f96ae940700 1 -- 172.21.15.62:0/3982737371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980095f0 con 0x7f96a81018f0 2024-05-08T22:35:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.988+0000 7f96ae940700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a81018f0 0x7f96a819e110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96a0007ee0 tx=0x7f96a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.988+0000 7f969f7fe700 1 -- 172.21.15.62:0/3982737371 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0017070 con 0x7f96a81018f0 2024-05-08T22:35:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.988+0000 7f969f7fe700 1 -- 172.21.15.62:0/3982737371 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96a000f040 con 0x7f96a81018f0 2024-05-08T22:35:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.988+0000 7f96b03a3700 1 -- 172.21.15.62:0/3982737371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a819eec0 con 0x7f96a81018f0 2024-05-08T22:35:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.988+0000 7f969f7fe700 1 -- 172.21.15.62:0/3982737371 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0005580 con 0x7f96a81018f0 2024-05-08T22:35:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.988+0000 7f96b03a3700 1 -- 172.21.15.62:0/3982737371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a819f340 con 0x7f96a81018f0 2024-05-08T22:35:13.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.989+0000 7f96b03a3700 1 -- 172.21.15.62:0/3982737371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a8107180 con 0x7f96a81018f0 2024-05-08T22:35:13.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.990+0000 7f969f7fe700 1 -- 172.21.15.62:0/3982737371 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f96a000db20 con 0x7f96a81018f0 2024-05-08T22:35:13.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.990+0000 7f969f7fe700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f96940a11a0 0x7f96940a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:13.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.991+0000 7f96ae13f700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f96940a11a0 0x7f96940a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:13.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.991+0000 7f969f7fe700 1 -- 172.21.15.62:0/3982737371 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f96a00cc470 con 0x7f96a81018f0 2024-05-08T22:35:13.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.991+0000 7f96ae13f700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f96940a11a0 0x7f96940a3660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9698003730 tx=0x7f9698000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:13.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:13.994+0000 7f969f7fe700 1 -- 172.21.15.62:0/3982737371 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f96a0095d50 con 0x7f96a81018f0 2024-05-08T22:35:14.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:13 smithi066 ceph-mon[46830]: pgmap v223: 97 pgs: 19 creating+peering, 64 active+clean, 14 unknown; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T22:35:14.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:13 smithi066 ceph-mon[46830]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T22:35:14.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:13 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2019814224' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T22:35:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:13 smithi066 ceph-mon[46830]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T22:35:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:13 smithi066 ceph-mon[46830]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:35:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:13 smithi066 ceph-mon[46830]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-08T22:35:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:13 smithi066 ceph-mon[46830]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-08T22:35:14.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:13 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1823056131' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T22:35:14.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:13 smithi079 ceph-mon[46738]: pgmap v223: 97 pgs: 19 creating+peering, 64 active+clean, 14 unknown; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T22:35:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:13 smithi079 ceph-mon[46738]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T22:35:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:13 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2019814224' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T22:35:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:13 smithi079 ceph-mon[46738]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T22:35:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:13 smithi079 ceph-mon[46738]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:35:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:13 smithi079 ceph-mon[46738]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-08T22:35:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:13 smithi079 ceph-mon[46738]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-08T22:35:14.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:13 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1823056131' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T22:35:14.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:13 smithi062 ceph-mon[42571]: pgmap v223: 97 pgs: 19 creating+peering, 64 active+clean, 14 unknown; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T22:35:14.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:13 smithi062 ceph-mon[42571]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T22:35:14.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:13 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2019814224' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T22:35:14.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:13 smithi062 ceph-mon[42571]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T22:35:14.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:13 smithi062 ceph-mon[42571]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T22:35:14.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:13 smithi062 ceph-mon[42571]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-08T22:35:14.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:13 smithi062 ceph-mon[42571]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-08T22:35:14.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:13 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1823056131' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T22:35:14.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.249+0000 7f96b03a3700 1 -- 172.21.15.62:0/3982737371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f96a8067d70 con 0x7f96a81018f0 2024-05-08T22:35:14.823 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:35:14 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-b[83604]: 2024-05-08T22:35:14.346+0000 7f0ce2184700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T22:35:14.823 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:35:14 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-b[83604]: 2024-05-08T22:35:14.346+0000 7f0ce2184700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T22:35:14.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.843+0000 7f969f7fe700 1 -- 172.21.15.62:0/3982737371 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f96a0099370 con 0x7f96a81018f0 2024-05-08T22:35:14.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.846+0000 7f96b03a3700 1 -- 172.21.15.62:0/3982737371 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f96940a11a0 msgr2=0x7f96940a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.846+0000 7f96b03a3700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f96940a11a0 0x7f96940a3660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9698003730 tx=0x7f9698000bc0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.846+0000 7f96b03a3700 1 -- 172.21.15.62:0/3982737371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a81018f0 msgr2=0x7f96a819e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:14.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.846+0000 7f96b03a3700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a81018f0 0x7f96a819e110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96a0007ee0 tx=0x7f96a000d3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.847+0000 7f96b03a3700 1 -- 172.21.15.62:0/3982737371 shutdown_connections 2024-05-08T22:35:14.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.847+0000 7f96b03a3700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96a81018f0 0x7f96a819e110 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.847+0000 7f96b03a3700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f96940a11a0 0x7f96940a3660 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.847+0000 7f96b03a3700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a80ffd60 0x7f96a8068be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.847+0000 7f96b03a3700 1 --2- 172.21.15.62:0/3982737371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96a8100f50 0x7f96a819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:14.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.847+0000 7f96b03a3700 1 -- 172.21.15.62:0/3982737371 >> 172.21.15.62:0/3982737371 conn(0x7f96a80fb290 msgr2=0x7f96a81042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:14.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.847+0000 7f96b03a3700 1 -- 172.21.15.62:0/3982737371 shutdown_connections 2024-05-08T22:35:14.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:14.847+0000 7f96b03a3700 1 -- 172.21.15.62:0/3982737371 wait complete. 2024-05-08T22:35:14.865 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-08T22:35:14.865 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-08T22:35:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.002+0000 7f9b3937e700 1 -- 172.21.15.62:0/2896756075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b340ffd60 msgr2=0x7f9b34100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.002+0000 7f9b3937e700 1 --2- 172.21.15.62:0/2896756075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b340ffd60 0x7f9b34100180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b240099c0 tx=0x7f9b24009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.002+0000 7f9b3937e700 1 -- 172.21.15.62:0/2896756075 shutdown_connections 2024-05-08T22:35:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.002+0000 7f9b3937e700 1 --2- 172.21.15.62:0/2896756075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b341018f0 0x7f9b341081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.002+0000 7f9b3937e700 1 --2- 172.21.15.62:0/2896756075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b340ffd60 0x7f9b34100180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.002+0000 7f9b3937e700 1 --2- 172.21.15.62:0/2896756075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b34100f50 0x7f9b341013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.002+0000 7f9b3937e700 1 -- 172.21.15.62:0/2896756075 >> 172.21.15.62:0/2896756075 conn(0x7f9b340fb2d0 msgr2=0x7f9b340fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.002+0000 7f9b3937e700 1 -- 172.21.15.62:0/2896756075 shutdown_connections 2024-05-08T22:35:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.002+0000 7f9b3937e700 1 -- 172.21.15.62:0/2896756075 wait complete. 2024-05-08T22:35:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b3937e700 1 Processor -- start 2024-05-08T22:35:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b3937e700 1 -- start start 2024-05-08T22:35:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b3937e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b340ffd60 0x7f9b3406a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b3937e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b34100f50 0x7f9b3419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b3937e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b341018f0 0x7f9b3419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b3937e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3419e8d0 con 0x7f9b341018f0 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b3937e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3419ea40 con 0x7f9b34100f50 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b3937e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3419ec00 con 0x7f9b340ffd60 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b341018f0 0x7f9b3419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b341018f0 0x7f9b3419e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48426/0 (socket says 172.21.15.62:48426) 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b337fe700 1 -- 172.21.15.62:0/3292282466 learned_addr learned my addr 172.21.15.62:0/3292282466 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b327fc700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b34100f50 0x7f9b3419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b32ffd700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b340ffd60 0x7f9b3406a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b337fe700 1 -- 172.21.15.62:0/3292282466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b340ffd60 msgr2=0x7f9b3406a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b337fe700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b340ffd60 0x7f9b3406a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b337fe700 1 -- 172.21.15.62:0/3292282466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b34100f50 msgr2=0x7f9b3419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b337fe700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b34100f50 0x7f9b3419bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b337fe700 1 -- 172.21.15.62:0/3292282466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b240095f0 con 0x7f9b341018f0 2024-05-08T22:35:15.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.003+0000 7f9b327fc700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b34100f50 0x7f9b3419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:15.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.004+0000 7f9b337fe700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b341018f0 0x7f9b3419e130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00ef30 tx=0x7f9b2c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.004+0000 7f9b1bfff700 1 -- 172.21.15.62:0/3292282466 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c018070 con 0x7f9b341018f0 2024-05-08T22:35:15.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.004+0000 7f9b1bfff700 1 -- 172.21.15.62:0/3292282466 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2c00f040 con 0x7f9b341018f0 2024-05-08T22:35:15.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.004+0000 7f9b3937e700 1 -- 172.21.15.62:0/3292282466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3419ee80 con 0x7f9b341018f0 2024-05-08T22:35:15.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.004+0000 7f9b1bfff700 1 -- 172.21.15.62:0/3292282466 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c003dc0 con 0x7f9b341018f0 2024-05-08T22:35:15.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.004+0000 7f9b3937e700 1 -- 172.21.15.62:0/3292282466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3419f300 con 0x7f9b341018f0 2024-05-08T22:35:15.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.005+0000 7f9b3937e700 1 -- 172.21.15.62:0/3292282466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b34107230 con 0x7f9b341018f0 2024-05-08T22:35:15.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.007+0000 7f9b1bfff700 1 -- 172.21.15.62:0/3292282466 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9b2c00caa0 con 0x7f9b341018f0 2024-05-08T22:35:15.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.007+0000 7f9b1bfff700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9b1c0a1120 0x7f9b1c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.007+0000 7f9b32ffd700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9b1c0a1120 0x7f9b1c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.007+0000 7f9b1bfff700 1 -- 172.21.15.62:0/3292282466 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9b2c0ccde0 con 0x7f9b341018f0 2024-05-08T22:35:15.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.008+0000 7f9b32ffd700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9b1c0a1120 0x7f9b1c0a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009990 tx=0x7f9b2401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.010+0000 7f9b1bfff700 1 -- 172.21.15.62:0/3292282466 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9b2c096750 con 0x7f9b341018f0 2024-05-08T22:35:15.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:14 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] up:active 2024-05-08T22:35:15.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:14 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1823056131' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T22:35:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:14 smithi079 ceph-mon[46738]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T22:35:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:14 smithi079 ceph-mon[46738]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-08T22:35:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:14 smithi079 ceph-mon[46738]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-08T22:35:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:14 smithi079 ceph-mon[46738]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-08T22:35:15.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:14 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3982737371' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T22:35:15.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:14 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] up:active 2024-05-08T22:35:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:14 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1823056131' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T22:35:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:14 smithi062 ceph-mon[42571]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T22:35:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:14 smithi062 ceph-mon[42571]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-08T22:35:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:14 smithi062 ceph-mon[42571]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-08T22:35:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:14 smithi062 ceph-mon[42571]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-08T22:35:15.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:14 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3982737371' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T22:35:15.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.248+0000 7f9b3937e700 1 -- 172.21.15.62:0/3292282466 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b3402ceb0 con 0x7f9b1c0a1120 2024-05-08T22:35:15.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.284+0000 7f9b1bfff700 1 -- 172.21.15.62:0/3292282466 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9b3402ceb0 con 0x7f9b1c0a1120 2024-05-08T22:35:15.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 -- 172.21.15.62:0/3292282466 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9b1c0a1120 msgr2=0x7f9b1c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9b1c0a1120 0x7f9b1c0a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009990 tx=0x7f9b2401a040 comp rx=0 tx=0).stop 2024-05-08T22:35:15.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 -- 172.21.15.62:0/3292282466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b341018f0 msgr2=0x7f9b3419e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b341018f0 0x7f9b3419e130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00ef30 tx=0x7f9b2c00c450 comp rx=0 tx=0).stop 2024-05-08T22:35:15.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 -- 172.21.15.62:0/3292282466 shutdown_connections 2024-05-08T22:35:15.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b341018f0 0x7f9b3419e130 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b340ffd60 0x7f9b3406a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9b1c0a1120 0x7f9b1c0a35e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 --2- 172.21.15.62:0/3292282466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b34100f50 0x7f9b3419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 -- 172.21.15.62:0/3292282466 >> 172.21.15.62:0/3292282466 conn(0x7f9b340fb2d0 msgr2=0x7f9b34104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:15.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 -- 172.21.15.62:0/3292282466 shutdown_connections 2024-05-08T22:35:15.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.287+0000 7f9b3937e700 1 -- 172.21.15.62:0/3292282466 wait complete. 2024-05-08T22:35:15.298 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-08T22:35:15.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:14 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] up:active 2024-05-08T22:35:15.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:14 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1823056131' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T22:35:15.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:14 smithi066 ceph-mon[46830]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T22:35:15.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:14 smithi066 ceph-mon[46830]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-08T22:35:15.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:14 smithi066 ceph-mon[46830]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-08T22:35:15.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:14 smithi066 ceph-mon[46830]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-08T22:35:15.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:14 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3982737371' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T22:35:15.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.431+0000 7fba68a4b700 1 -- 172.21.15.62:0/1619709440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba64108d60 msgr2=0x7fba6410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.431+0000 7fba68a4b700 1 --2- 172.21.15.62:0/1619709440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba64108d60 0x7fba6410b1f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00b0a0 tx=0x7fba5c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.431+0000 7fba68a4b700 1 -- 172.21.15.62:0/1619709440 shutdown_connections 2024-05-08T22:35:15.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.431+0000 7fba68a4b700 1 --2- 172.21.15.62:0/1619709440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba64108d60 0x7fba6410b1f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.431+0000 7fba68a4b700 1 --2- 172.21.15.62:0/1619709440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba64074f60 0x7fba640733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.431+0000 7fba68a4b700 1 --2- 172.21.15.62:0/1619709440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba64073980 0x7fba64073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.431+0000 7fba68a4b700 1 -- 172.21.15.62:0/1619709440 >> 172.21.15.62:0/1619709440 conn(0x7fba640fc400 msgr2=0x7fba640fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.431+0000 7fba68a4b700 1 -- 172.21.15.62:0/1619709440 shutdown_connections 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.431+0000 7fba68a4b700 1 -- 172.21.15.62:0/1619709440 wait complete. 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba68a4b700 1 Processor -- start 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba68a4b700 1 -- start start 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba68a4b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba64073980 0x7fba64199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba68a4b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba64074f60 0x7fba64199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba68a4b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba64108d60 0x7fba641a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba68a4b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a0b80 con 0x7fba64073980 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba68a4b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a0cc0 con 0x7fba64108d60 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba68a4b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a0e50 con 0x7fba64074f60 2024-05-08T22:35:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba6259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba64073980 0x7fba64199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba6259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba64073980 0x7fba64199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48470/0 (socket says 172.21.15.62:48470) 2024-05-08T22:35:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba6259c700 1 -- 172.21.15.62:0/967801415 learned_addr learned my addr 172.21.15.62:0/967801415 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba61d9b700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba64074f60 0x7fba64199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba62d9d700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba64108d60 0x7fba641a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba6259c700 1 -- 172.21.15.62:0/967801415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba64074f60 msgr2=0x7fba64199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba6259c700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba64074f60 0x7fba64199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba6259c700 1 -- 172.21.15.62:0/967801415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba64108d60 msgr2=0x7fba641a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba6259c700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba64108d60 0x7fba641a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba6259c700 1 -- 172.21.15.62:0/967801415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c009d00 con 0x7fba64073980 2024-05-08T22:35:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.432+0000 7fba61d9b700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba64074f60 0x7fba64199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.433+0000 7fba6259c700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba64073980 0x7fba64199050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba5800e9a0 tx=0x7fba5800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.433+0000 7fba537fe700 1 -- 172.21.15.62:0/967801415 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58009790 con 0x7fba64073980 2024-05-08T22:35:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.433+0000 7fba68a4b700 1 -- 172.21.15.62:0/967801415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba641a1130 con 0x7fba64073980 2024-05-08T22:35:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.433+0000 7fba537fe700 1 -- 172.21.15.62:0/967801415 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba5800cbd0 con 0x7fba64073980 2024-05-08T22:35:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.433+0000 7fba537fe700 1 -- 172.21.15.62:0/967801415 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58018680 con 0x7fba64073980 2024-05-08T22:35:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.433+0000 7fba68a4b700 1 -- 172.21.15.62:0/967801415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba641a1640 con 0x7fba64073980 2024-05-08T22:35:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.434+0000 7fba68a4b700 1 -- 172.21.15.62:0/967801415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba64107b70 con 0x7fba64073980 2024-05-08T22:35:15.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.435+0000 7fba537fe700 1 -- 172.21.15.62:0/967801415 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fba5800cd40 con 0x7fba64073980 2024-05-08T22:35:15.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.435+0000 7fba537fe700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba4c0a1120 0x7fba4c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.436+0000 7fba61d9b700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba4c0a1120 0x7fba4c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.436+0000 7fba537fe700 1 -- 172.21.15.62:0/967801415 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fba58014070 con 0x7fba64073980 2024-05-08T22:35:15.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.436+0000 7fba61d9b700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba4c0a1120 0x7fba4c0a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba54005950 tx=0x7fba540058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.439+0000 7fba537fe700 1 -- 172.21.15.62:0/967801415 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fba58096860 con 0x7fba64073980 2024-05-08T22:35:15.671 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:35:15 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:35:15.256+0000 7f676b5d1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:35:15.672 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:35:15 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:35:15.256+0000 7f676b5d1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:35:15.672 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:35:15 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:35:15.256+0000 7f676b5d1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:35:15.672 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:35:15 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:35:15.256+0000 7f676b5d1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:35:15.672 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:35:15 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:35:15.256+0000 7f676b5d1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T22:35:15.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.684+0000 7fba68a4b700 1 -- 172.21.15.62:0/967801415 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fba6402ce10 con 0x7fba4c0a1120 2024-05-08T22:35:15.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.705+0000 7fba537fe700 1 -- 172.21.15.62:0/967801415 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fba6402ce10 con 0x7fba4c0a1120 2024-05-08T22:35:15.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 -- 172.21.15.62:0/967801415 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba4c0a1120 msgr2=0x7fba4c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba4c0a1120 0x7fba4c0a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba54005950 tx=0x7fba540058e0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 -- 172.21.15.62:0/967801415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba64073980 msgr2=0x7fba64199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba64073980 0x7fba64199050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba5800e9a0 tx=0x7fba5800ed60 comp rx=0 tx=0).stop 2024-05-08T22:35:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 -- 172.21.15.62:0/967801415 shutdown_connections 2024-05-08T22:35:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba4c0a1120 0x7fba4c0a35e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba64108d60 0x7fba641a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba64073980 0x7fba64199050 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 --2- 172.21.15.62:0/967801415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba64074f60 0x7fba64199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 -- 172.21.15.62:0/967801415 >> 172.21.15.62:0/967801415 conn(0x7fba640fc400 msgr2=0x7fba64104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 -- 172.21.15.62:0/967801415 shutdown_connections 2024-05-08T22:35:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.707+0000 7fba68a4b700 1 -- 172.21.15.62:0/967801415 wait complete. 2024-05-08T22:35:15.719 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-08T22:35:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.851+0000 7ff28879b700 1 -- 172.21.15.62:0/2361244623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2800ffdc0 msgr2=0x7ff2801001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.851+0000 7ff28879b700 1 --2- 172.21.15.62:0/2361244623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2800ffdc0 0x7ff2801001e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0099c0 tx=0x7ff27c009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 -- 172.21.15.62:0/2361244623 shutdown_connections 2024-05-08T22:35:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 --2- 172.21.15.62:0/2361244623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff280101950 0x7ff280108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 --2- 172.21.15.62:0/2361244623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2800ffdc0 0x7ff2801001e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 --2- 172.21.15.62:0/2361244623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff280100fb0 0x7ff280101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 -- 172.21.15.62:0/2361244623 >> 172.21.15.62:0/2361244623 conn(0x7ff2800fb330 msgr2=0x7ff2800fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 -- 172.21.15.62:0/2361244623 shutdown_connections 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 -- 172.21.15.62:0/2361244623 wait complete. 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 Processor -- start 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 -- start start 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2800ffdc0 0x7ff280103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff280100fb0 0x7ff280102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff280101950 0x7ff2801028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff280102fd0 con 0x7ff280100fb0 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff280103140 con 0x7ff2800ffdc0 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff28879b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2801a0f30 con 0x7ff280101950 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.852+0000 7ff285d36700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff280100fb0 0x7ff280102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff285d36700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff280100fb0 0x7ff280102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48498/0 (socket says 172.21.15.62:48498) 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff285d36700 1 -- 172.21.15.62:0/3885066210 learned_addr learned my addr 172.21.15.62:0/3885066210 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff286d38700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff280101950 0x7ff2801028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff286537700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2800ffdc0 0x7ff280103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff285d36700 1 -- 172.21.15.62:0/3885066210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff280101950 msgr2=0x7ff2801028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff285d36700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff280101950 0x7ff2801028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff285d36700 1 -- 172.21.15.62:0/3885066210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2800ffdc0 msgr2=0x7ff280103cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff285d36700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2800ffdc0 0x7ff280103cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff285d36700 1 -- 172.21.15.62:0/3885066210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c0095f0 con 0x7ff280100fb0 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff285d36700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff280100fb0 0x7ff280102390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff27000d980 tx=0x7ff27000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff2777fe700 1 -- 172.21.15.62:0/3885066210 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270004220 con 0x7ff280100fb0 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff28879b700 1 -- 172.21.15.62:0/3885066210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2801a1190 con 0x7ff280100fb0 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff2777fe700 1 -- 172.21.15.62:0/3885066210 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff270007640 con 0x7ff280100fb0 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff2777fe700 1 -- 172.21.15.62:0/3885066210 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270005690 con 0x7ff280100fb0 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.853+0000 7ff28879b700 1 -- 172.21.15.62:0/3885066210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2801a16a0 con 0x7ff280100fb0 2024-05-08T22:35:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.854+0000 7ff28879b700 1 -- 172.21.15.62:0/3885066210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff280067480 con 0x7ff280100fb0 2024-05-08T22:35:15.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.856+0000 7ff2777fe700 1 -- 172.21.15.62:0/3885066210 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff270004750 con 0x7ff280100fb0 2024-05-08T22:35:15.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.856+0000 7ff2777fe700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff26c0a1120 0x7ff26c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:15.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.857+0000 7ff2777fe700 1 -- 172.21.15.62:0/3885066210 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff2700ccb70 con 0x7ff280100fb0 2024-05-08T22:35:15.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.857+0000 7ff286537700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff26c0a1120 0x7ff26c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:15.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.857+0000 7ff286537700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff26c0a1120 0x7ff26c0a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009f50 tx=0x7ff27c005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:15.860+0000 7ff2777fe700 1 -- 172.21.15.62:0/3885066210 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff2700963b0 con 0x7ff280100fb0 2024-05-08T22:35:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.105+0000 7ff28879b700 1 -- 172.21.15.62:0/3885066210 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7ff280069750 con 0x7ff26c0a1120 2024-05-08T22:35:16.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.109+0000 7ff2777fe700 1 -- 172.21.15.62:0/3885066210 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7ff280069750 con 0x7ff26c0a1120 2024-05-08T22:35:16.108 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/qa/sv_0/623c3bde-4d0d-43b0-b0c7-bae55f90416f 2024-05-08T22:35:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 -- 172.21.15.62:0/3885066210 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff26c0a1120 msgr2=0x7ff26c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff26c0a1120 0x7ff26c0a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009f50 tx=0x7ff27c005ea0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 -- 172.21.15.62:0/3885066210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff280100fb0 msgr2=0x7ff280102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff280100fb0 0x7ff280102390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff27000d980 tx=0x7ff27000dc90 comp rx=0 tx=0).stop 2024-05-08T22:35:16.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 -- 172.21.15.62:0/3885066210 shutdown_connections 2024-05-08T22:35:16.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff280101950 0x7ff2801028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff26c0a1120 0x7ff26c0a35e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2800ffdc0 0x7ff280103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 --2- 172.21.15.62:0/3885066210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff280100fb0 0x7ff280102390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 -- 172.21.15.62:0/3885066210 >> 172.21.15.62:0/3885066210 conn(0x7ff2800fb330 msgr2=0x7ff280069040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:16.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 -- 172.21.15.62:0/3885066210 shutdown_connections 2024-05-08T22:35:16.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.111+0000 7ff28879b700 1 -- 172.21.15.62:0/3885066210 wait complete. 2024-05-08T22:35:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: pgmap v224: 97 pgs: 16 creating+peering, 77 active+clean, 4 unknown; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 4 op/s 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] up:active 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3982737371' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] up:active 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: Cluster is now healthy 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-05-08T22:35:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-08T22:35:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:15 smithi062 ceph-mon[42571]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-05-08T22:35:16.123 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-08T22:35:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: pgmap v224: 97 pgs: 16 creating+peering, 77 active+clean, 4 unknown; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 4 op/s 2024-05-08T22:35:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] up:active 2024-05-08T22:35:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3982737371' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T22:35:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T22:35:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-08T22:35:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-05-08T22:35:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-05-08T22:35:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] up:active 2024-05-08T22:35:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T22:35:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T22:35:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: Cluster is now healthy 2024-05-08T22:35:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-05-08T22:35:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-08T22:35:16.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:15 smithi079 ceph-mon[46738]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-05-08T22:35:16.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.257+0000 7f82481e0700 1 -- 172.21.15.62:0/1907752691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8240102980 msgr2=0x7f82401092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.257+0000 7f82481e0700 1 --2- 172.21.15.62:0/1907752691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8240102980 0x7f82401092a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f823800b0a0 tx=0x7f823800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.257+0000 7f82481e0700 1 -- 172.21.15.62:0/1907752691 shutdown_connections 2024-05-08T22:35:16.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.257+0000 7f82481e0700 1 --2- 172.21.15.62:0/1907752691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8240102980 0x7f82401092a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.257+0000 7f82481e0700 1 --2- 172.21.15.62:0/1907752691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8240100e70 0x7f8240101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.257+0000 7f82481e0700 1 --2- 172.21.15.62:0/1907752691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8240101fc0 0x7f8240102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.257+0000 7f82481e0700 1 -- 172.21.15.62:0/1907752691 >> 172.21.15.62:0/1907752691 conn(0x7f82400fc400 msgr2=0x7f82400fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.257+0000 7f82481e0700 1 -- 172.21.15.62:0/1907752691 shutdown_connections 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.258+0000 7f82481e0700 1 -- 172.21.15.62:0/1907752691 wait complete. 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.258+0000 7f82481e0700 1 Processor -- start 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.258+0000 7f82481e0700 1 -- start start 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.258+0000 7f82481e0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8240100e70 0x7f8240074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.258+0000 7f82481e0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8240101fc0 0x7f8240073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.258+0000 7f82481e0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8240102980 0x7f82400738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.258+0000 7f82481e0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8240074070 con 0x7f8240102980 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f82481e0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82400741e0 con 0x7f8240101fc0 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f82481e0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82401a0ed0 con 0x7f8240100e70 2024-05-08T22:35:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f824677d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8240102980 0x7f82400738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:16.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f824577b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8240101fc0 0x7f8240073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:16.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f824677d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8240102980 0x7f82400738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48522/0 (socket says 172.21.15.62:48522) 2024-05-08T22:35:16.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f824677d700 1 -- 172.21.15.62:0/3043879473 learned_addr learned my addr 172.21.15.62:0/3043879473 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:16.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f8245f7c700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8240100e70 0x7f8240074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:16.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f824677d700 1 -- 172.21.15.62:0/3043879473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8240100e70 msgr2=0x7f8240074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f824677d700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8240100e70 0x7f8240074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f824677d700 1 -- 172.21.15.62:0/3043879473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8240101fc0 msgr2=0x7f8240073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f824677d700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8240101fc0 0x7f8240073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f824677d700 1 -- 172.21.15.62:0/3043879473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8238009d00 con 0x7f8240102980 2024-05-08T22:35:16.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.259+0000 7f824677d700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8240102980 0x7f82400738d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8238006070 tx=0x7f8238004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:16.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.262+0000 7f8236ffd700 1 -- 172.21.15.62:0/3043879473 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823800e040 con 0x7f8240102980 2024-05-08T22:35:16.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.262+0000 7f82481e0700 1 -- 172.21.15.62:0/3043879473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82401a1100 con 0x7f8240102980 2024-05-08T22:35:16.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.262+0000 7f82481e0700 1 -- 172.21.15.62:0/3043879473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82401a1580 con 0x7f8240102980 2024-05-08T22:35:16.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.263+0000 7f82481e0700 1 -- 172.21.15.62:0/3043879473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8240108210 con 0x7f8240102980 2024-05-08T22:35:16.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.263+0000 7f8236ffd700 1 -- 172.21.15.62:0/3043879473 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8238009b80 con 0x7f8240102980 2024-05-08T22:35:16.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.263+0000 7f8236ffd700 1 -- 172.21.15.62:0/3043879473 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823801b8d0 con 0x7f8240102980 2024-05-08T22:35:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.265+0000 7f8236ffd700 1 -- 172.21.15.62:0/3043879473 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f823801ba30 con 0x7f8240102980 2024-05-08T22:35:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.265+0000 7f8236ffd700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f822c0a1120 0x7f822c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.265+0000 7f8245f7c700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f822c0a1120 0x7f822c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.265+0000 7f8236ffd700 1 -- 172.21.15.62:0/3043879473 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f82380ceba0 con 0x7f8240102980 2024-05-08T22:35:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.266+0000 7f8245f7c700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f822c0a1120 0x7f822c0a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8230007c60 tx=0x7f82300073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.269+0000 7f8236ffd700 1 -- 172.21.15.62:0/3043879473 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f82380cee60 con 0x7f8240102980 2024-05-08T22:35:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.281+0000 7f8236ffd700 1 -- 172.21.15.62:0/3043879473 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f823801bce0 con 0x7f8240102980 2024-05-08T22:35:16.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: pgmap v224: 97 pgs: 16 creating+peering, 77 active+clean, 4 unknown; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 4 op/s 2024-05-08T22:35:16.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] up:active 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3982737371' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] up:active 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: Cluster is now healthy 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-08T22:35:16.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:15 smithi066 ceph-mon[46830]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-05-08T22:35:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.519+0000 7f82481e0700 1 -- 172.21.15.62:0/3043879473 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f824002ce10 con 0x7f822c0a1120 2024-05-08T22:35:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.542+0000 7f8236ffd700 1 -- 172.21.15.62:0/3043879473 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f824002ce10 con 0x7f822c0a1120 2024-05-08T22:35:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 -- 172.21.15.62:0/3043879473 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f822c0a1120 msgr2=0x7f822c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f822c0a1120 0x7f822c0a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8230007c60 tx=0x7f82300073d0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 -- 172.21.15.62:0/3043879473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8240102980 msgr2=0x7f82400738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8240102980 0x7f82400738d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8238006070 tx=0x7f8238004d00 comp rx=0 tx=0).stop 2024-05-08T22:35:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 -- 172.21.15.62:0/3043879473 shutdown_connections 2024-05-08T22:35:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8240102980 0x7f82400738d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8240100e70 0x7f8240074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f822c0a1120 0x7f822c0a35e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 --2- 172.21.15.62:0/3043879473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8240101fc0 0x7f8240073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 -- 172.21.15.62:0/3043879473 >> 172.21.15.62:0/3043879473 conn(0x7f82400fc400 msgr2=0x7f8240105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 -- 172.21.15.62:0/3043879473 shutdown_connections 2024-05-08T22:35:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.544+0000 7f82481e0700 1 -- 172.21.15.62:0/3043879473 wait complete. 2024-05-08T22:35:16.555 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-08T22:35:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.687+0000 7fb741923700 1 -- 172.21.15.62:0/518818147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c101980 msgr2=0x7fb73c103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.687+0000 7fb741923700 1 --2- 172.21.15.62:0/518818147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c101980 0x7fb73c103d70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb730009990 tx=0x7fb730009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.688+0000 7fb741923700 1 -- 172.21.15.62:0/518818147 shutdown_connections 2024-05-08T22:35:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.688+0000 7fb741923700 1 --2- 172.21.15.62:0/518818147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c106be0 0x7fb73c108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.688+0000 7fb741923700 1 --2- 172.21.15.62:0/518818147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c101980 0x7fb73c103d70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.688+0000 7fb741923700 1 --2- 172.21.15.62:0/518818147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb73c1042b0 0x7fb73c1066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.688+0000 7fb741923700 1 -- 172.21.15.62:0/518818147 >> 172.21.15.62:0/518818147 conn(0x7fb73c0fb310 msgr2=0x7fb73c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.688+0000 7fb741923700 1 -- 172.21.15.62:0/518818147 shutdown_connections 2024-05-08T22:35:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.688+0000 7fb741923700 1 -- 172.21.15.62:0/518818147 wait complete. 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.688+0000 7fb741923700 1 Processor -- start 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.688+0000 7fb741923700 1 -- start start 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.688+0000 7fb741923700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c101980 0x7fb73c196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb741923700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb73c1042b0 0x7fb73c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb741923700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c106be0 0x7fb73c19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb741923700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c19e850 con 0x7fb73c101980 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb741923700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c19e9c0 con 0x7fb73c106be0 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb741923700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c19eb80 con 0x7fb73c1042b0 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c101980 0x7fb73c196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c101980 0x7fb73c196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60426/0 (socket says 172.21.15.62:60426) 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73affd700 1 -- 172.21.15.62:0/3368353636 learned_addr learned my addr 172.21.15.62:0/3368353636 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73a7fc700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb73c1042b0 0x7fb73c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73b7fe700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c106be0 0x7fb73c19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73affd700 1 -- 172.21.15.62:0/3368353636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb73c1042b0 msgr2=0x7fb73c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73affd700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb73c1042b0 0x7fb73c1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73affd700 1 -- 172.21.15.62:0/3368353636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c106be0 msgr2=0x7fb73c19e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73affd700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c106be0 0x7fb73c19e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73affd700 1 -- 172.21.15.62:0/3368353636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300095f0 con 0x7fb73c101980 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73a7fc700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb73c1042b0 0x7fb73c1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb73affd700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c101980 0x7fb73c196d60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb7300095c0 tx=0x7fb73000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb740921700 1 -- 172.21.15.62:0/3368353636 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73001d070 con 0x7fb73c101980 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb740921700 1 -- 172.21.15.62:0/3368353636 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb73000fb80 con 0x7fb73c101980 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb741923700 1 -- 172.21.15.62:0/3368353636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73c19ee00 con 0x7fb73c101980 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb740921700 1 -- 172.21.15.62:0/3368353636 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730017650 con 0x7fb73c101980 2024-05-08T22:35:16.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.689+0000 7fb741923700 1 -- 172.21.15.62:0/3368353636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73c19f1d0 con 0x7fb73c101980 2024-05-08T22:35:16.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.690+0000 7fb741923700 1 -- 172.21.15.62:0/3368353636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73c0fce90 con 0x7fb73c101980 2024-05-08T22:35:16.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.691+0000 7fb740921700 1 -- 172.21.15.62:0/3368353636 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb73000fcf0 con 0x7fb73c101980 2024-05-08T22:35:16.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.692+0000 7fb740921700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7240a11b0 0x7fb7240a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:16.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.692+0000 7fb73a7fc700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7240a11b0 0x7fb7240a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:16.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.692+0000 7fb740921700 1 -- 172.21.15.62:0/3368353636 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb7300cdef0 con 0x7fb73c101980 2024-05-08T22:35:16.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.692+0000 7fb73a7fc700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7240a11b0 0x7fb7240a3670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb72c005950 tx=0x7fb72c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:16.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.695+0000 7fb740921700 1 -- 172.21.15.62:0/3368353636 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb730097660 con 0x7fb73c101980 2024-05-08T22:35:16.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.943+0000 7fb741923700 1 -- 172.21.15.62:0/3368353636 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb73c02ceb0 con 0x7fb7240a11b0 2024-05-08T22:35:16.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.947+0000 7fb740921700 1 -- 172.21.15.62:0/3368353636 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fb73c02ceb0 con 0x7fb7240a11b0 2024-05-08T22:35:16.946 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/qa/sv_1/511d854d-9d6f-47bd-ba9a-35f51f2d3dc7 2024-05-08T22:35:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 -- 172.21.15.62:0/3368353636 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7240a11b0 msgr2=0x7fb7240a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7240a11b0 0x7fb7240a3670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb72c005950 tx=0x7fb72c0058e0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 -- 172.21.15.62:0/3368353636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c101980 msgr2=0x7fb73c196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c101980 0x7fb73c196d60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb7300095c0 tx=0x7fb73000f690 comp rx=0 tx=0).stop 2024-05-08T22:35:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 -- 172.21.15.62:0/3368353636 shutdown_connections 2024-05-08T22:35:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7240a11b0 0x7fb7240a3670 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb73c106be0 0x7fb73c19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c101980 0x7fb73c196d60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 --2- 172.21.15.62:0/3368353636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb73c1042b0 0x7fb73c1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 -- 172.21.15.62:0/3368353636 >> 172.21.15.62:0/3368353636 conn(0x7fb73c0fb310 msgr2=0x7fb73c102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 -- 172.21.15.62:0/3368353636 shutdown_connections 2024-05-08T22:35:16.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:16.949+0000 7fb741923700 1 -- 172.21.15.62:0/3368353636 wait complete. 2024-05-08T22:35:16.960 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.088+0000 7f3df9266700 1 -- 172.21.15.62:0/1102632817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4101910 msgr2=0x7f3df41081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.088+0000 7f3df9266700 1 --2- 172.21.15.62:0/1102632817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4101910 0x7f3df41081c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00b0d0 tx=0x7f3dec00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.088+0000 7f3df9266700 1 -- 172.21.15.62:0/1102632817 shutdown_connections 2024-05-08T22:35:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.088+0000 7f3df9266700 1 --2- 172.21.15.62:0/1102632817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4101910 0x7f3df41081c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.088+0000 7f3df9266700 1 --2- 172.21.15.62:0/1102632817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df40ffd80 0x7f3df41001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.088+0000 7f3df9266700 1 --2- 172.21.15.62:0/1102632817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df4100f70 0x7f3df41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.088+0000 7f3df9266700 1 -- 172.21.15.62:0/1102632817 >> 172.21.15.62:0/1102632817 conn(0x7f3df40fb310 msgr2=0x7f3df40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.088+0000 7f3df9266700 1 -- 172.21.15.62:0/1102632817 shutdown_connections 2024-05-08T22:35:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.088+0000 7f3df9266700 1 -- 172.21.15.62:0/1102632817 wait complete. 2024-05-08T22:35:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.089+0000 7f3df9266700 1 Processor -- start 2024-05-08T22:35:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.089+0000 7f3df9266700 1 -- start start 2024-05-08T22:35:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.089+0000 7f3df9266700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df40ffd80 0x7f3df4196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.089+0000 7f3df9266700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df4100f70 0x7f3df41973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.089+0000 7f3df9266700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df4101910 0x7f3df419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df9266700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df419e8e0 con 0x7f3df40ffd80 2024-05-08T22:35:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df9266700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df419ea50 con 0x7f3df4101910 2024-05-08T22:35:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df9266700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df419ec10 con 0x7f3df4100f70 2024-05-08T22:35:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df40ffd80 0x7f3df4196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df40ffd80 0x7f3df4196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60450/0 (socket says 172.21.15.62:60450) 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df2ffd700 1 -- 172.21.15.62:0/850508515 learned_addr learned my addr 172.21.15.62:0/850508515 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df37fe700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df4101910 0x7f3df419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df27fc700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df4100f70 0x7f3df41973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df2ffd700 1 -- 172.21.15.62:0/850508515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df4100f70 msgr2=0x7f3df41973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df2ffd700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df4100f70 0x7f3df41973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df2ffd700 1 -- 172.21.15.62:0/850508515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df4101910 msgr2=0x7f3df419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df2ffd700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df4101910 0x7f3df419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df2ffd700 1 -- 172.21.15.62:0/850508515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec009d00 con 0x7f3df40ffd80 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df2ffd700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df40ffd80 0x7f3df4196e80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3de400c990 tx=0x7f3de400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3ddbfff700 1 -- 172.21.15.62:0/850508515 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4007940 con 0x7f3df40ffd80 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3ddbfff700 1 -- 172.21.15.62:0/850508515 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de400f450 con 0x7f3df40ffd80 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df9266700 1 -- 172.21.15.62:0/850508515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df419eef0 con 0x7f3df40ffd80 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3ddbfff700 1 -- 172.21.15.62:0/850508515 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4018660 con 0x7f3df40ffd80 2024-05-08T22:35:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.090+0000 7f3df9266700 1 -- 172.21.15.62:0/850508515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df419f310 con 0x7f3df40ffd80 2024-05-08T22:35:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.091+0000 7f3df9266700 1 -- 172.21.15.62:0/850508515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df40681d0 con 0x7f3df40ffd80 2024-05-08T22:35:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.092+0000 7f3ddbfff700 1 -- 172.21.15.62:0/850508515 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3de400f830 con 0x7f3df40ffd80 2024-05-08T22:35:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.093+0000 7f3ddbfff700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3ddc0a11b0 0x7f3ddc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.093+0000 7f3df27fc700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3ddc0a11b0 0x7f3ddc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.093+0000 7f3ddbfff700 1 -- 172.21.15.62:0/850508515 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3de40cc7d0 con 0x7f3df40ffd80 2024-05-08T22:35:17.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.093+0000 7f3df27fc700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3ddc0a11b0 0x7f3ddc0a3670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3df4198270 tx=0x7f3de800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:17.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.097+0000 7f3ddbfff700 1 -- 172.21.15.62:0/850508515 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3df40681d0 con 0x7f3df40ffd80 2024-05-08T22:35:17.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:16 smithi079 ceph-mon[46738]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:17.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:16 smithi079 ceph-mon[46738]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:17.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:16 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] up:active 2024-05-08T22:35:17.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:16 smithi079 ceph-mon[46738]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-08T22:35:17.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:16 smithi079 ceph-mon[46738]: mgrmap e16: x(active, since 5m), standbys: z, y 2024-05-08T22:35:17.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:16 smithi079 ceph-mon[46738]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:35:17.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:16 smithi062 ceph-mon[42571]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:17.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:16 smithi062 ceph-mon[42571]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:17.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:16 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] up:active 2024-05-08T22:35:17.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:16 smithi062 ceph-mon[42571]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-08T22:35:17.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:16 smithi062 ceph-mon[42571]: mgrmap e16: x(active, since 5m), standbys: z, y 2024-05-08T22:35:17.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:16 smithi062 ceph-mon[42571]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:35:17.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:16 smithi066 ceph-mon[46830]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:17.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:16 smithi066 ceph-mon[46830]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:17.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:16 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] up:active 2024-05-08T22:35:17.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:16 smithi066 ceph-mon[46830]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-08T22:35:17.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:16 smithi066 ceph-mon[46830]: mgrmap e16: x(active, since 5m), standbys: z, y 2024-05-08T22:35:17.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:16 smithi066 ceph-mon[46830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:35:17.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.355+0000 7f3df9266700 1 -- 172.21.15.62:0/850508515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3df410b5a0 con 0x7f3df40ffd80 2024-05-08T22:35:17.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.355+0000 7f3ddbfff700 1 -- 172.21.15.62:0/850508515 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10897 (secure 0 0 0) 0x7f3de4095fb0 con 0x7f3df40ffd80 2024-05-08T22:35:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:17.355 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:35:09.803459+0000","modified":"2024-05-08T22:35:16.261557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24529,"mds_2":24505,"mds_3":24503,"mds_4":14661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/1756743060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1756743060},{"type":"v1","addr":"172.21.15.62:6839","nonce":1756743060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14661":{"gid":14661,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/780017086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":780017086},{"type":"v1","addr":"172.21.15.62:6841","nonce":780017086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.79:6839/3534785027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":3534785027},{"type":"v1","addr":"172.21.15.79:6839","nonce":3534785027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/3474672912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":3474672912},{"type":"v1","addr":"172.21.15.66:6839","nonce":3474672912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.357+0000 7f3df9266700 1 -- 172.21.15.62:0/850508515 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3ddc0a11b0 msgr2=0x7f3ddc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.357+0000 7f3df9266700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3ddc0a11b0 0x7f3ddc0a3670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3df4198270 tx=0x7f3de800a380 comp rx=0 tx=0).stop 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.357+0000 7f3df9266700 1 -- 172.21.15.62:0/850508515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df40ffd80 msgr2=0x7f3df4196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.357+0000 7f3df9266700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df40ffd80 0x7f3df4196e80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3de400c990 tx=0x7f3de400cca0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.358+0000 7f3df9266700 1 -- 172.21.15.62:0/850508515 shutdown_connections 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.358+0000 7f3df9266700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3ddc0a11b0 0x7f3ddc0a3670 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.358+0000 7f3df9266700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df4101910 0x7f3df419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.358+0000 7f3df9266700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df40ffd80 0x7f3df4196e80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.358+0000 7f3df9266700 1 --2- 172.21.15.62:0/850508515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3df4100f70 0x7f3df41973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.358+0000 7f3df9266700 1 -- 172.21.15.62:0/850508515 >> 172.21.15.62:0/850508515 conn(0x7f3df40fb310 msgr2=0x7f3df40fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.358+0000 7f3df9266700 1 -- 172.21.15.62:0/850508515 shutdown_connections 2024-05-08T22:35:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.358+0000 7f3df9266700 1 -- 172.21.15.62:0/850508515 wait complete. 2024-05-08T22:35:17.358 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-05-08T22:35:17.369 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:17.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.498+0000 7f3ed0954700 1 -- 172.21.15.62:0/1326568446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ecc104230 msgr2=0x7f3ecc106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.498+0000 7f3ed0954700 1 --2- 172.21.15.62:0/1326568446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ecc104230 0x7f3ecc106620 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009990 tx=0x7f3ebc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.498+0000 7f3ed0954700 1 -- 172.21.15.62:0/1326568446 shutdown_connections 2024-05-08T22:35:17.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.498+0000 7f3ed0954700 1 --2- 172.21.15.62:0/1326568446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ecc106b60 0x7f3ecc108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.498+0000 7f3ed0954700 1 --2- 172.21.15.62:0/1326568446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc101900 0x7f3ecc103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.498+0000 7f3ed0954700 1 --2- 172.21.15.62:0/1326568446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ecc104230 0x7f3ecc106620 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.498+0000 7f3ed0954700 1 -- 172.21.15.62:0/1326568446 >> 172.21.15.62:0/1326568446 conn(0x7f3ecc0fb310 msgr2=0x7f3ecc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.498+0000 7f3ed0954700 1 -- 172.21.15.62:0/1326568446 shutdown_connections 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.498+0000 7f3ed0954700 1 -- 172.21.15.62:0/1326568446 wait complete. 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3ed0954700 1 Processor -- start 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3ed0954700 1 -- start start 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3ed0954700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ecc101900 0x7f3ecc101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3ed0954700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ecc104230 0x7f3ecc0ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3ed0954700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc106b60 0x7f3ecc0ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3ed0954700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc100590 con 0x7f3ecc101900 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3ed0954700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc100700 con 0x7f3ecc104230 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3ed0954700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc1a0e80 con 0x7f3ecc106b60 2024-05-08T22:35:17.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3eca59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ecc101900 0x7f3ecc101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3eca59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ecc101900 0x7f3ecc101220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60478/0 (socket says 172.21.15.62:60478) 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3eca59c700 1 -- 172.21.15.62:0/1068461851 learned_addr learned my addr 172.21.15.62:0/1068461851 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3ecad9d700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc106b60 0x7f3ecc0ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3ec9d9b700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ecc104230 0x7f3ecc0ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3eca59c700 1 -- 172.21.15.62:0/1068461851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc106b60 msgr2=0x7f3ecc0ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3eca59c700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc106b60 0x7f3ecc0ffdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3eca59c700 1 -- 172.21.15.62:0/1068461851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ecc104230 msgr2=0x7f3ecc0ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3eca59c700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ecc104230 0x7f3ecc0ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.499+0000 7f3eca59c700 1 -- 172.21.15.62:0/1068461851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc0095f0 con 0x7f3ecc101900 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.500+0000 7f3eca59c700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ecc101900 0x7f3ecc101220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000d980 tx=0x7f3ec000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.500+0000 7f3ebb7fe700 1 -- 172.21.15.62:0/1068461851 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0004220 con 0x7f3ecc101900 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.500+0000 7f3ed0954700 1 -- 172.21.15.62:0/1068461851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc1a1110 con 0x7f3ecc101900 2024-05-08T22:35:17.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.500+0000 7f3ebb7fe700 1 -- 172.21.15.62:0/1068461851 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ec0007640 con 0x7f3ecc101900 2024-05-08T22:35:17.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.500+0000 7f3ebb7fe700 1 -- 172.21.15.62:0/1068461851 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0004e20 con 0x7f3ecc101900 2024-05-08T22:35:17.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.500+0000 7f3ed0954700 1 -- 172.21.15.62:0/1068461851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc1a1620 con 0x7f3ecc101900 2024-05-08T22:35:17.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.501+0000 7f3ed0954700 1 -- 172.21.15.62:0/1068461851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ecc0fce90 con 0x7f3ecc101900 2024-05-08T22:35:17.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.502+0000 7f3ebb7fe700 1 -- 172.21.15.62:0/1068461851 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3ec000b900 con 0x7f3ecc101900 2024-05-08T22:35:17.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.502+0000 7f3ebb7fe700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3eb40a11b0 0x7f3eb40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.503+0000 7f3ec9d9b700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3eb40a11b0 0x7f3eb40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.503+0000 7f3ebb7fe700 1 -- 172.21.15.62:0/1068461851 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3ec00cc8d0 con 0x7f3ecc101900 2024-05-08T22:35:17.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.503+0000 7f3ec9d9b700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3eb40a11b0 0x7f3eb40a3670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00b7b0 tx=0x7f3ebc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:17.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.507+0000 7f3ebb7fe700 1 -- 172.21.15.62:0/1068461851 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3ec0095fb0 con 0x7f3ecc101900 2024-05-08T22:35:17.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.717+0000 7f3ed0954700 1 -- 172.21.15.62:0/1068461851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ecc068160 con 0x7f3ecc101900 2024-05-08T22:35:17.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.719+0000 7f3ebb7fe700 1 -- 172.21.15.62:0/1068461851 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22047 (secure 0 0 0) 0x7f3ec0016020 con 0x7f3ecc101900 2024-05-08T22:35:17.718 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:17.718 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":79,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:35:10.805199+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:17.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.721+0000 7f3ed0954700 1 -- 172.21.15.62:0/1068461851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3eb40a11b0 msgr2=0x7f3eb40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.721+0000 7f3ed0954700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3eb40a11b0 0x7f3eb40a3670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00b7b0 tx=0x7f3ebc005fd0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.721+0000 7f3ed0954700 1 -- 172.21.15.62:0/1068461851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ecc101900 msgr2=0x7f3ecc101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.721+0000 7f3ed0954700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ecc101900 0x7f3ecc101220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000d980 tx=0x7f3ec000dd40 comp rx=0 tx=0).stop 2024-05-08T22:35:17.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.721+0000 7f3ed0954700 1 -- 172.21.15.62:0/1068461851 shutdown_connections 2024-05-08T22:35:17.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.721+0000 7f3ed0954700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ecc106b60 0x7f3ecc0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.721+0000 7f3ed0954700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ecc101900 0x7f3ecc101220 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.721+0000 7f3ed0954700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3eb40a11b0 0x7f3eb40a3670 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.721+0000 7f3ed0954700 1 --2- 172.21.15.62:0/1068461851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ecc104230 0x7f3ecc0ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.721+0000 7f3ed0954700 1 -- 172.21.15.62:0/1068461851 >> 172.21.15.62:0/1068461851 conn(0x7f3ecc0fb310 msgr2=0x7f3ecc1037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:17.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.722+0000 7f3ed0954700 1 -- 172.21.15.62:0/1068461851 shutdown_connections 2024-05-08T22:35:17.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.722+0000 7f3ed0954700 1 -- 172.21.15.62:0/1068461851 wait complete. 2024-05-08T22:35:17.733 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T22:35:17.733 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:35:17.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.864+0000 7facb0191700 1 -- 172.21.15.62:0/2018740693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faca8102990 msgr2=0x7faca81092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.864+0000 7facb0191700 1 --2- 172.21.15.62:0/2018740693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faca8102990 0x7faca81092b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faca000b0a0 tx=0x7faca000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.864+0000 7facb0191700 1 -- 172.21.15.62:0/2018740693 shutdown_connections 2024-05-08T22:35:17.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.864+0000 7facb0191700 1 --2- 172.21.15.62:0/2018740693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faca8102990 0x7faca81092b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.864+0000 7facb0191700 1 --2- 172.21.15.62:0/2018740693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faca8100e80 0x7faca81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.864+0000 7facb0191700 1 --2- 172.21.15.62:0/2018740693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca8101fd0 0x7faca8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.864+0000 7facb0191700 1 -- 172.21.15.62:0/2018740693 >> 172.21.15.62:0/2018740693 conn(0x7faca80fc410 msgr2=0x7faca80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:17.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.864+0000 7facb0191700 1 -- 172.21.15.62:0/2018740693 shutdown_connections 2024-05-08T22:35:17.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.864+0000 7facb0191700 1 -- 172.21.15.62:0/2018740693 wait complete. 2024-05-08T22:35:17.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.864+0000 7facb0191700 1 Processor -- start 2024-05-08T22:35:17.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.864+0000 7facb0191700 1 -- start start 2024-05-08T22:35:17.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facb0191700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faca8100e80 0x7faca8196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facb0191700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faca8101fd0 0x7faca81973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facb0191700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca8102990 0x7faca819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facb0191700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca819e9e0 con 0x7faca8101fd0 2024-05-08T22:35:17.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facb0191700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca819eb20 con 0x7faca8102990 2024-05-08T22:35:17.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facb0191700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca819ecb0 con 0x7faca8100e80 2024-05-08T22:35:17.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facad72c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faca8101fd0 0x7faca81973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facad72c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faca8101fd0 0x7faca81973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60492/0 (socket says 172.21.15.62:60492) 2024-05-08T22:35:17.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facad72c700 1 -- 172.21.15.62:0/968152454 learned_addr learned my addr 172.21.15.62:0/968152454 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facadf2d700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faca8100e80 0x7faca8196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facae72e700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca8102990 0x7faca819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facad72c700 1 -- 172.21.15.62:0/968152454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faca8100e80 msgr2=0x7faca8196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facad72c700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faca8100e80 0x7faca8196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facad72c700 1 -- 172.21.15.62:0/968152454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca8102990 msgr2=0x7faca819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facad72c700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca8102990 0x7faca819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facad72c700 1 -- 172.21.15.62:0/968152454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca0009d00 con 0x7faca8101fd0 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facad72c700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faca8101fd0 0x7faca81973f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac9800d950 tx=0x7fac9800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7fac9effd700 1 -- 172.21.15.62:0/968152454 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9800f890 con 0x7faca8101fd0 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7fac9effd700 1 -- 172.21.15.62:0/968152454 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac98010460 con 0x7faca8101fd0 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.865+0000 7facb0191700 1 -- 172.21.15.62:0/968152454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca819ef90 con 0x7faca8101fd0 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.866+0000 7fac9effd700 1 -- 172.21.15.62:0/968152454 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9800e610 con 0x7faca8101fd0 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.866+0000 7facb0191700 1 -- 172.21.15.62:0/968152454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca819f410 con 0x7faca8101fd0 2024-05-08T22:35:17.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.866+0000 7facb0191700 1 -- 172.21.15.62:0/968152454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca8108220 con 0x7faca8101fd0 2024-05-08T22:35:17.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.867+0000 7fac9effd700 1 -- 172.21.15.62:0/968152454 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fac980108d0 con 0x7faca8101fd0 2024-05-08T22:35:17.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.868+0000 7fac9effd700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fac940a11b0 0x7fac940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:17.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.868+0000 7facadf2d700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fac940a11b0 0x7fac940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:17.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.868+0000 7fac9effd700 1 -- 172.21.15.62:0/968152454 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fac980cc9d0 con 0x7faca8101fd0 2024-05-08T22:35:17.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.868+0000 7facadf2d700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fac940a11b0 0x7fac940a3670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faca4005950 tx=0x7faca400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:17.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:17.872+0000 7fac9effd700 1 -- 172.21.15.62:0/968152454 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fac980961b0 con 0x7faca8101fd0 2024-05-08T22:35:18.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.079+0000 7facb0191700 1 -- 172.21.15.62:0/968152454 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faca802ceb0 con 0x7fac940a11b0 2024-05-08T22:35:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.089+0000 7fac9effd700 1 -- 172.21.15.62:0/968152454 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+328600 (secure 0 0 0) 0x7faca802ceb0 con 0x7fac940a11b0 2024-05-08T22:35:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:18.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 -- 172.21.15.62:0/968152454 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fac940a11b0 msgr2=0x7fac940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fac940a11b0 0x7fac940a3670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faca4005950 tx=0x7faca400a300 comp rx=0 tx=0).stop 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 -- 172.21.15.62:0/968152454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faca8101fd0 msgr2=0x7faca81973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faca8101fd0 0x7faca81973f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac9800d950 tx=0x7fac9800dc60 comp rx=0 tx=0).stop 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 -- 172.21.15.62:0/968152454 shutdown_connections 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fac940a11b0 0x7fac940a3670 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faca8102990 0x7faca819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faca8100e80 0x7faca8196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 --2- 172.21.15.62:0/968152454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faca8101fd0 0x7faca81973f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 -- 172.21.15.62:0/968152454 >> 172.21.15.62:0/968152454 conn(0x7faca80fc410 msgr2=0x7faca8105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 -- 172.21.15.62:0/968152454 shutdown_connections 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.093+0000 7facb0191700 1 -- 172.21.15.62:0/968152454 wait complete. 2024-05-08T22:35:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-05-08T22:35:18.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:17 smithi062 ceph-mon[42571]: pgmap v225: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 154 B/s rd, 6.0 KiB/s wr, 15 op/s 2024-05-08T22:35:18.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:17 smithi062 ceph-mon[42571]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:18.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:17 smithi062 ceph-mon[42571]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:18.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:17 smithi062 ceph-mon[42571]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:18.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:17 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/850508515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:18.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:17 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1068461851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:18.105 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-05-08T22:35:18.010888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633980,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":150,"num_read_kb":146,"num_write":352,"num_write_kb":3034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":288,"ondisk_log_size":288,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381288,"kb_used_data":12028,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124560536,"statfs":{"total":1151940427776,"available":1151549988864,"internally_reserved":0,"allocated":12316672,"data_stored":6741045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":377992518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43612,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":24,"num_read_kb":37,"num_write":171,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.203332"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.840865+0000","last_change":"2024-05-08T22:35:09.840453+0000","last_active":"2024-05-08T22:35:09.840865+0000","last_peered":"2024-05-08T22:35:09.840865+0000","last_clean":"2024-05-08T22:35:09.840865+0000","last_became_active":"2024-05-08T22:35:09.840093+0000","last_became_peered":"2024-05-08T22:35:09.840093+0000","last_unstale":"2024-05-08T22:35:09.840865+0000","last_undegraded":"2024-05-08T22:35:09.840865+0000","last_fullsized":"2024-05-08T22:35:09.840865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:13.265286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840996+0000","last_change":"2024-05-08T22:35:09.832735+0000","last_active":"2024-05-08T22:35:11.840996+0000","last_peered":"2024-05-08T22:35:11.840996+0000","last_clean":"2024-05-08T22:35:11.840996+0000","last_became_active":"2024-05-08T22:35:09.832419+0000","last_became_peered":"2024-05-08T22:35:09.832419+0000","last_unstale":"2024-05-08T22:35:11.840996+0000","last_undegraded":"2024-05-08T22:35:11.840996+0000","last_fullsized":"2024-05-08T22:35:11.840996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:19.711398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841216+0000","last_change":"2024-05-08T22:35:09.835604+0000","last_active":"2024-05-08T22:35:11.841216+0000","last_peered":"2024-05-08T22:35:11.841216+0000","last_clean":"2024-05-08T22:35:11.841216+0000","last_became_active":"2024-05-08T22:35:09.835235+0000","last_became_peered":"2024-05-08T22:35:09.835235+0000","last_unstale":"2024-05-08T22:35:11.841216+0000","last_undegraded":"2024-05-08T22:35:11.841216+0000","last_fullsized":"2024-05-08T22:35:11.841216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:35:10.722947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.831424+0000","last_change":"2024-05-08T22:35:09.824876+0000","last_active":"2024-05-08T22:35:09.831424+0000","last_peered":"2024-05-08T22:35:09.831424+0000","last_clean":"2024-05-08T22:35:09.831424+0000","last_became_active":"2024-05-08T22:35:09.823494+0000","last_became_peered":"2024-05-08T22:35:09.823494+0000","last_unstale":"2024-05-08T22:35:09.831424+0000","last_undegraded":"2024-05-08T22:35:09.831424+0000","last_fullsized":"2024-05-08T22:35:09.831424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:20.099947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.828965+0000","last_change":"2024-05-08T22:35:09.825647+0000","last_active":"2024-05-08T22:35:09.828965+0000","last_peered":"2024-05-08T22:35:09.828965+0000","last_clean":"2024-05-08T22:35:09.828965+0000","last_became_active":"2024-05-08T22:35:09.825322+0000","last_became_peered":"2024-05-08T22:35:09.825322+0000","last_unstale":"2024-05-08T22:35:09.828965+0000","last_undegraded":"2024-05-08T22:35:09.828965+0000","last_fullsized":"2024-05-08T22:35:09.828965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:29.080541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604425+0000","last_change":"2024-05-08T22:35:09.841894+0000","last_active":"2024-05-08T22:35:11.604425+0000","last_peered":"2024-05-08T22:35:11.604425+0000","last_clean":"2024-05-08T22:35:11.604425+0000","last_became_active":"2024-05-08T22:35:09.841429+0000","last_became_peered":"2024-05-08T22:35:09.841429+0000","last_unstale":"2024-05-08T22:35:11.604425+0000","last_undegraded":"2024-05-08T22:35:11.604425+0000","last_fullsized":"2024-05-08T22:35:11.604425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:50.859186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.622990+0000","last_change":"2024-05-08T22:35:09.834010+0000","last_active":"2024-05-08T22:35:11.622990+0000","last_peered":"2024-05-08T22:35:11.622990+0000","last_clean":"2024-05-08T22:35:11.622990+0000","last_became_active":"2024-05-08T22:35:09.833633+0000","last_became_peered":"2024-05-08T22:35:09.833633+0000","last_unstale":"2024-05-08T22:35:11.622990+0000","last_undegraded":"2024-05-08T22:35:11.622990+0000","last_fullsized":"2024-05-08T22:35:11.622990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:28.854383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825479+0000","last_change":"2024-05-08T22:35:09.820817+0000","last_active":"2024-05-08T22:35:11.825479+0000","last_peered":"2024-05-08T22:35:11.825479+0000","last_clean":"2024-05-08T22:35:11.825479+0000","last_became_active":"2024-05-08T22:35:09.820521+0000","last_became_peered":"2024-05-08T22:35:09.820521+0000","last_unstale":"2024-05-08T22:35:11.825479+0000","last_undegraded":"2024-05-08T22:35:11.825479+0000","last_fullsized":"2024-05-08T22:35:11.825479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:17.312794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812985+0000","last_change":"2024-05-08T22:35:09.838821+0000","last_active":"2024-05-08T22:35:10.812985+0000","last_peered":"2024-05-08T22:35:10.812985+0000","last_clean":"2024-05-08T22:35:10.812985+0000","last_became_active":"2024-05-08T22:35:09.836588+0000","last_became_peered":"2024-05-08T22:35:09.836588+0000","last_unstale":"2024-05-08T22:35:10.812985+0000","last_undegraded":"2024-05-08T22:35:10.812985+0000","last_fullsized":"2024-05-08T22:35:10.812985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:52.884827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604696+0000","last_change":"2024-05-08T22:35:09.836222+0000","last_active":"2024-05-08T22:35:11.604696+0000","last_peered":"2024-05-08T22:35:11.604696+0000","last_clean":"2024-05-08T22:35:11.604696+0000","last_became_active":"2024-05-08T22:35:09.835810+0000","last_became_peered":"2024-05-08T22:35:09.835810+0000","last_unstale":"2024-05-08T22:35:11.604696+0000","last_undegraded":"2024-05-08T22:35:11.604696+0000","last_fullsized":"2024-05-08T22:35:11.604696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:38:14.075576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623061+0000","last_change":"2024-05-08T22:35:09.835422+0000","last_active":"2024-05-08T22:35:11.623061+0000","last_peered":"2024-05-08T22:35:11.623061+0000","last_clean":"2024-05-08T22:35:11.623061+0000","last_became_active":"2024-05-08T22:35:09.835100+0000","last_became_peered":"2024-05-08T22:35:09.835100+0000","last_unstale":"2024-05-08T22:35:11.623061+0000","last_undegraded":"2024-05-08T22:35:11.623061+0000","last_fullsized":"2024-05-08T22:35:11.623061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:12.099286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.832873+0000","last_change":"2024-05-08T22:35:09.827007+0000","last_active":"2024-05-08T22:35:09.832873+0000","last_peered":"2024-05-08T22:35:09.832873+0000","last_clean":"2024-05-08T22:35:09.832873+0000","last_became_active":"2024-05-08T22:35:09.826639+0000","last_became_peered":"2024-05-08T22:35:09.826639+0000","last_unstale":"2024-05-08T22:35:09.832873+0000","last_undegraded":"2024-05-08T22:35:09.832873+0000","last_fullsized":"2024-05-08T22:35:09.832873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:39.999049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470041+0000","last_change":"2024-05-08T22:35:09.828507+0000","last_active":"2024-05-08T22:35:10.470041+0000","last_peered":"2024-05-08T22:35:10.470041+0000","last_clean":"2024-05-08T22:35:10.470041+0000","last_became_active":"2024-05-08T22:35:09.828076+0000","last_became_peered":"2024-05-08T22:35:09.828076+0000","last_unstale":"2024-05-08T22:35:10.470041+0000","last_undegraded":"2024-05-08T22:35:10.470041+0000","last_fullsized":"2024-05-08T22:35:10.470041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:33.475538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840770+0000","last_change":"2024-05-08T22:35:09.820506+0000","last_active":"2024-05-08T22:35:11.840770+0000","last_peered":"2024-05-08T22:35:11.840770+0000","last_clean":"2024-05-08T22:35:11.840770+0000","last_became_active":"2024-05-08T22:35:09.820208+0000","last_became_peered":"2024-05-08T22:35:09.820208+0000","last_unstale":"2024-05-08T22:35:11.840770+0000","last_undegraded":"2024-05-08T22:35:11.840770+0000","last_fullsized":"2024-05-08T22:35:11.840770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:47:00.753509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840848+0000","last_change":"2024-05-08T22:35:09.837171+0000","last_active":"2024-05-08T22:35:11.840848+0000","last_peered":"2024-05-08T22:35:11.840848+0000","last_clean":"2024-05-08T22:35:11.840848+0000","last_became_active":"2024-05-08T22:35:09.836290+0000","last_became_peered":"2024-05-08T22:35:09.836290+0000","last_unstale":"2024-05-08T22:35:11.840848+0000","last_undegraded":"2024-05-08T22:35:11.840848+0000","last_fullsized":"2024-05-08T22:35:11.840848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:13.029182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812403+0000","last_change":"2024-05-08T22:35:09.836148+0000","last_active":"2024-05-08T22:35:10.812403+0000","last_peered":"2024-05-08T22:35:10.812403+0000","last_clean":"2024-05-08T22:35:10.812403+0000","last_became_active":"2024-05-08T22:35:09.835767+0000","last_became_peered":"2024-05-08T22:35:09.835767+0000","last_unstale":"2024-05-08T22:35:10.812403+0000","last_undegraded":"2024-05-08T22:35:10.812403+0000","last_fullsized":"2024-05-08T22:35:10.812403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:23.343495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.841116+0000","last_change":"2024-05-08T22:35:09.840558+0000","last_active":"2024-05-08T22:35:09.841116+0000","last_peered":"2024-05-08T22:35:09.841116+0000","last_clean":"2024-05-08T22:35:09.841116+0000","last_became_active":"2024-05-08T22:35:09.840080+0000","last_became_peered":"2024-05-08T22:35:09.840080+0000","last_unstale":"2024-05-08T22:35:09.841116+0000","last_undegraded":"2024-05-08T22:35:09.841116+0000","last_fullsized":"2024-05-08T22:35:09.841116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:32.081217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.842689+0000","last_change":"2024-05-08T22:35:09.831151+0000","last_active":"2024-05-08T22:35:11.842689+0000","last_peered":"2024-05-08T22:35:11.842689+0000","last_clean":"2024-05-08T22:35:11.842689+0000","last_became_active":"2024-05-08T22:35:09.830693+0000","last_became_peered":"2024-05-08T22:35:09.830693+0000","last_unstale":"2024-05-08T22:35:11.842689+0000","last_undegraded":"2024-05-08T22:35:11.842689+0000","last_fullsized":"2024-05-08T22:35:11.842689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:15.060787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841080+0000","last_change":"2024-05-08T22:35:09.830620+0000","last_active":"2024-05-08T22:35:11.841080+0000","last_peered":"2024-05-08T22:35:11.841080+0000","last_clean":"2024-05-08T22:35:11.841080+0000","last_became_active":"2024-05-08T22:35:09.830289+0000","last_became_peered":"2024-05-08T22:35:09.830289+0000","last_unstale":"2024-05-08T22:35:11.841080+0000","last_undegraded":"2024-05-08T22:35:11.841080+0000","last_fullsized":"2024-05-08T22:35:11.841080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:49:15.181957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.847355+0000","last_change":"2024-05-08T22:35:09.391570+0000","last_active":"2024-05-08T22:35:13.847355+0000","last_peered":"2024-05-08T22:35:13.847355+0000","last_clean":"2024-05-08T22:35:13.847355+0000","last_became_active":"2024-05-08T22:35:09.391234+0000","last_became_peered":"2024-05-08T22:35:09.391234+0000","last_unstale":"2024-05-08T22:35:13.847355+0000","last_undegraded":"2024-05-08T22:35:13.847355+0000","last_fullsized":"2024-05-08T22:35:13.847355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:37.226688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825589+0000","last_change":"2024-05-08T22:35:09.831973+0000","last_active":"2024-05-08T22:35:11.825589+0000","last_peered":"2024-05-08T22:35:11.825589+0000","last_clean":"2024-05-08T22:35:11.825589+0000","last_became_active":"2024-05-08T22:35:09.829817+0000","last_became_peered":"2024-05-08T22:35:09.829817+0000","last_unstale":"2024-05-08T22:35:11.825589+0000","last_undegraded":"2024-05-08T22:35:11.825589+0000","last_fullsized":"2024-05-08T22:35:11.825589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:53.829103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841201+0000","last_change":"2024-05-08T22:35:09.393821+0000","last_active":"2024-05-08T22:35:11.841201+0000","last_peered":"2024-05-08T22:35:11.841201+0000","last_clean":"2024-05-08T22:35:11.841201+0000","last_became_active":"2024-05-08T22:35:09.393503+0000","last_became_peered":"2024-05-08T22:35:09.393503+0000","last_unstale":"2024-05-08T22:35:11.841201+0000","last_undegraded":"2024-05-08T22:35:11.841201+0000","last_fullsized":"2024-05-08T22:35:11.841201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:56.252015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841407+0000","last_change":"2024-05-08T22:35:09.823341+0000","last_active":"2024-05-08T22:35:11.841407+0000","last_peered":"2024-05-08T22:35:11.841407+0000","last_clean":"2024-05-08T22:35:11.841407+0000","last_became_active":"2024-05-08T22:35:09.822949+0000","last_became_peered":"2024-05-08T22:35:09.822949+0000","last_unstale":"2024-05-08T22:35:11.841407+0000","last_undegraded":"2024-05-08T22:35:11.841407+0000","last_fullsized":"2024-05-08T22:35:11.841407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:49.189563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.845906+0000","last_change":"2024-05-08T22:35:09.391129+0000","last_active":"2024-05-08T22:35:13.845906+0000","last_peered":"2024-05-08T22:35:13.845906+0000","last_clean":"2024-05-08T22:35:13.845906+0000","last_became_active":"2024-05-08T22:35:09.390429+0000","last_became_peered":"2024-05-08T22:35:09.390429+0000","last_unstale":"2024-05-08T22:35:13.845906+0000","last_undegraded":"2024-05-08T22:35:13.845906+0000","last_fullsized":"2024-05-08T22:35:13.845906+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:18.416456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812741+0000","last_change":"2024-05-08T22:35:09.838274+0000","last_active":"2024-05-08T22:35:10.812741+0000","last_peered":"2024-05-08T22:35:10.812741+0000","last_clean":"2024-05-08T22:35:10.812741+0000","last_became_active":"2024-05-08T22:35:09.837637+0000","last_became_peered":"2024-05-08T22:35:09.837637+0000","last_unstale":"2024-05-08T22:35:10.812741+0000","last_undegraded":"2024-05-08T22:35:10.812741+0000","last_fullsized":"2024-05-08T22:35:10.812741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:00.476914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.832187+0000","last_change":"2024-05-08T22:35:09.391987+0000","last_active":"2024-05-08T22:35:09.832187+0000","last_peered":"2024-05-08T22:35:09.832187+0000","last_clean":"2024-05-08T22:35:09.832187+0000","last_became_active":"2024-05-08T22:35:09.390587+0000","last_became_peered":"2024-05-08T22:35:09.390587+0000","last_unstale":"2024-05-08T22:35:09.832187+0000","last_undegraded":"2024-05-08T22:35:09.832187+0000","last_fullsized":"2024-05-08T22:35:09.832187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:14.343341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813185+0000","last_change":"2024-05-08T22:35:09.823177+0000","last_active":"2024-05-08T22:35:10.813185+0000","last_peered":"2024-05-08T22:35:10.813185+0000","last_clean":"2024-05-08T22:35:10.813185+0000","last_became_active":"2024-05-08T22:35:09.822830+0000","last_became_peered":"2024-05-08T22:35:09.822830+0000","last_unstale":"2024-05-08T22:35:10.813185+0000","last_undegraded":"2024-05-08T22:35:10.813185+0000","last_fullsized":"2024-05-08T22:35:10.813185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:44.648818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:16.264577+0000","last_change":"2024-05-08T22:35:09.392558+0000","last_active":"2024-05-08T22:35:16.264577+0000","last_peered":"2024-05-08T22:35:16.264577+0000","last_clean":"2024-05-08T22:35:16.264577+0000","last_became_active":"2024-05-08T22:35:09.392235+0000","last_became_peered":"2024-05-08T22:35:09.392235+0000","last_unstale":"2024-05-08T22:35:16.264577+0000","last_undegraded":"2024-05-08T22:35:16.264577+0000","last_fullsized":"2024-05-08T22:35:16.264577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:47:12.186400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.830402+0000","last_change":"2024-05-08T22:35:09.823455+0000","last_active":"2024-05-08T22:35:09.830402+0000","last_peered":"2024-05-08T22:35:09.830402+0000","last_clean":"2024-05-08T22:35:09.830402+0000","last_became_active":"2024-05-08T22:35:09.822818+0000","last_became_peered":"2024-05-08T22:35:09.822818+0000","last_unstale":"2024-05-08T22:35:09.830402+0000","last_undegraded":"2024-05-08T22:35:09.830402+0000","last_fullsized":"2024-05-08T22:35:09.830402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:38.142265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:14.857485+0000","last_change":"2024-05-08T22:35:09.394713+0000","last_active":"2024-05-08T22:35:14.857485+0000","last_peered":"2024-05-08T22:35:14.857485+0000","last_clean":"2024-05-08T22:35:14.857485+0000","last_became_active":"2024-05-08T22:35:09.394397+0000","last_became_peered":"2024-05-08T22:35:09.394397+0000","last_unstale":"2024-05-08T22:35:14.857485+0000","last_undegraded":"2024-05-08T22:35:14.857485+0000","last_fullsized":"2024-05-08T22:35:14.857485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:58:30.594207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813065+0000","last_change":"2024-05-08T22:35:09.827921+0000","last_active":"2024-05-08T22:35:10.813065+0000","last_peered":"2024-05-08T22:35:10.813065+0000","last_clean":"2024-05-08T22:35:10.813065+0000","last_became_active":"2024-05-08T22:35:09.827387+0000","last_became_peered":"2024-05-08T22:35:09.827387+0000","last_unstale":"2024-05-08T22:35:10.813065+0000","last_undegraded":"2024-05-08T22:35:10.813065+0000","last_fullsized":"2024-05-08T22:35:10.813065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:33.986935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.845741+0000","last_change":"2024-05-08T22:35:09.384387+0000","last_active":"2024-05-08T22:35:13.845741+0000","last_peered":"2024-05-08T22:35:13.845741+0000","last_clean":"2024-05-08T22:35:13.845741+0000","last_became_active":"2024-05-08T22:35:09.383970+0000","last_became_peered":"2024-05-08T22:35:09.383970+0000","last_unstale":"2024-05-08T22:35:13.845741+0000","last_undegraded":"2024-05-08T22:35:13.845741+0000","last_fullsized":"2024-05-08T22:35:13.845741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:55.430960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825444+0000","last_change":"2024-05-08T22:35:09.831477+0000","last_active":"2024-05-08T22:35:11.825444+0000","last_peered":"2024-05-08T22:35:11.825444+0000","last_clean":"2024-05-08T22:35:11.825444+0000","last_became_active":"2024-05-08T22:35:09.831168+0000","last_became_peered":"2024-05-08T22:35:09.831168+0000","last_unstale":"2024-05-08T22:35:11.825444+0000","last_undegraded":"2024-05-08T22:35:11.825444+0000","last_fullsized":"2024-05-08T22:35:11.825444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:56.765210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267651+0000","last_change":"2024-05-08T22:35:09.389063+0000","last_active":"2024-05-08T22:35:15.267651+0000","last_peered":"2024-05-08T22:35:15.267651+0000","last_clean":"2024-05-08T22:35:15.267651+0000","last_became_active":"2024-05-08T22:35:09.388683+0000","last_became_peered":"2024-05-08T22:35:09.388683+0000","last_unstale":"2024-05-08T22:35:15.267651+0000","last_undegraded":"2024-05-08T22:35:15.267651+0000","last_fullsized":"2024-05-08T22:35:15.267651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:03.804688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.842741+0000","last_change":"2024-05-08T22:35:09.829277+0000","last_active":"2024-05-08T22:35:11.842741+0000","last_peered":"2024-05-08T22:35:11.842741+0000","last_clean":"2024-05-08T22:35:11.842741+0000","last_became_active":"2024-05-08T22:35:09.828782+0000","last_became_peered":"2024-05-08T22:35:09.828782+0000","last_unstale":"2024-05-08T22:35:11.842741+0000","last_undegraded":"2024-05-08T22:35:11.842741+0000","last_fullsized":"2024-05-08T22:35:11.842741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:47.012528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:16.264245+0000","last_change":"2024-05-08T22:35:09.387464+0000","last_active":"2024-05-08T22:35:16.264245+0000","last_peered":"2024-05-08T22:35:16.264245+0000","last_clean":"2024-05-08T22:35:16.264245+0000","last_became_active":"2024-05-08T22:35:09.386994+0000","last_became_peered":"2024-05-08T22:35:09.386994+0000","last_unstale":"2024-05-08T22:35:16.264245+0000","last_undegraded":"2024-05-08T22:35:16.264245+0000","last_fullsized":"2024-05-08T22:35:16.264245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:27:04.151479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812918+0000","last_change":"2024-05-08T22:35:09.830886+0000","last_active":"2024-05-08T22:35:10.812918+0000","last_peered":"2024-05-08T22:35:10.812918+0000","last_clean":"2024-05-08T22:35:10.812918+0000","last_became_active":"2024-05-08T22:35:09.830466+0000","last_became_peered":"2024-05-08T22:35:09.830466+0000","last_unstale":"2024-05-08T22:35:10.812918+0000","last_undegraded":"2024-05-08T22:35:10.812918+0000","last_fullsized":"2024-05-08T22:35:10.812918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:12.337934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267954+0000","last_change":"2024-05-08T22:35:09.391510+0000","last_active":"2024-05-08T22:35:15.267954+0000","last_peered":"2024-05-08T22:35:15.267954+0000","last_clean":"2024-05-08T22:35:15.267954+0000","last_became_active":"2024-05-08T22:35:09.391145+0000","last_became_peered":"2024-05-08T22:35:09.391145+0000","last_unstale":"2024-05-08T22:35:15.267954+0000","last_undegraded":"2024-05-08T22:35:15.267954+0000","last_fullsized":"2024-05-08T22:35:15.267954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:39.694122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841304+0000","last_change":"2024-05-08T22:35:09.837056+0000","last_active":"2024-05-08T22:35:11.841304+0000","last_peered":"2024-05-08T22:35:11.841304+0000","last_clean":"2024-05-08T22:35:11.841304+0000","last_became_active":"2024-05-08T22:35:09.836724+0000","last_became_peered":"2024-05-08T22:35:09.836724+0000","last_unstale":"2024-05-08T22:35:11.841304+0000","last_undegraded":"2024-05-08T22:35:11.841304+0000","last_fullsized":"2024-05-08T22:35:11.841304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:48.901289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:14.855791+0000","last_change":"2024-05-08T22:35:09.395826+0000","last_active":"2024-05-08T22:35:14.855791+0000","last_peered":"2024-05-08T22:35:14.855791+0000","last_clean":"2024-05-08T22:35:14.855791+0000","last_became_active":"2024-05-08T22:35:09.395330+0000","last_became_peered":"2024-05-08T22:35:09.395330+0000","last_unstale":"2024-05-08T22:35:14.855791+0000","last_undegraded":"2024-05-08T22:35:14.855791+0000","last_fullsized":"2024-05-08T22:35:14.855791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:43.577894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812839+0000","last_change":"2024-05-08T22:35:09.831625+0000","last_active":"2024-05-08T22:35:10.812839+0000","last_peered":"2024-05-08T22:35:10.812839+0000","last_clean":"2024-05-08T22:35:10.812839+0000","last_became_active":"2024-05-08T22:35:09.831310+0000","last_became_peered":"2024-05-08T22:35:09.831310+0000","last_unstale":"2024-05-08T22:35:10.812839+0000","last_undegraded":"2024-05-08T22:35:10.812839+0000","last_fullsized":"2024-05-08T22:35:10.812839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:16.743871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'21","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:16.534895+0000","last_change":"2024-05-08T22:35:09.389805+0000","last_active":"2024-05-08T22:35:16.534895+0000","last_peered":"2024-05-08T22:35:16.534895+0000","last_clean":"2024-05-08T22:35:16.534895+0000","last_became_active":"2024-05-08T22:35:09.389443+0000","last_became_peered":"2024-05-08T22:35:09.389443+0000","last_unstale":"2024-05-08T22:35:16.534895+0000","last_undegraded":"2024-05-08T22:35:16.534895+0000","last_fullsized":"2024-05-08T22:35:16.534895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:40.479503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38147,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":23,"num_write":23,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.835300+0000","last_change":"2024-05-08T22:35:09.834871+0000","last_active":"2024-05-08T22:35:09.835300+0000","last_peered":"2024-05-08T22:35:09.835300+0000","last_clean":"2024-05-08T22:35:09.835300+0000","last_became_active":"2024-05-08T22:35:09.834589+0000","last_became_peered":"2024-05-08T22:35:09.834589+0000","last_unstale":"2024-05-08T22:35:09.835300+0000","last_undegraded":"2024-05-08T22:35:09.835300+0000","last_fullsized":"2024-05-08T22:35:09.835300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:12.476142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.844535+0000","last_change":"2024-05-08T22:35:09.388615+0000","last_active":"2024-05-08T22:35:13.844535+0000","last_peered":"2024-05-08T22:35:13.844535+0000","last_clean":"2024-05-08T22:35:13.844535+0000","last_became_active":"2024-05-08T22:35:09.388272+0000","last_became_peered":"2024-05-08T22:35:09.388272+0000","last_unstale":"2024-05-08T22:35:13.844535+0000","last_undegraded":"2024-05-08T22:35:13.844535+0000","last_fullsized":"2024-05-08T22:35:13.844535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:23.565450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470024+0000","last_change":"2024-05-08T22:35:09.831511+0000","last_active":"2024-05-08T22:35:10.470024+0000","last_peered":"2024-05-08T22:35:10.470024+0000","last_clean":"2024-05-08T22:35:10.470024+0000","last_became_active":"2024-05-08T22:35:09.831216+0000","last_became_peered":"2024-05-08T22:35:09.831216+0000","last_unstale":"2024-05-08T22:35:10.470024+0000","last_undegraded":"2024-05-08T22:35:10.470024+0000","last_fullsized":"2024-05-08T22:35:10.470024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:46.244544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.268190+0000","last_change":"2024-05-08T22:35:09.391835+0000","last_active":"2024-05-08T22:35:15.268190+0000","last_peered":"2024-05-08T22:35:15.268190+0000","last_clean":"2024-05-08T22:35:15.268190+0000","last_became_active":"2024-05-08T22:35:09.391465+0000","last_became_peered":"2024-05-08T22:35:09.391465+0000","last_unstale":"2024-05-08T22:35:15.268190+0000","last_undegraded":"2024-05-08T22:35:15.268190+0000","last_fullsized":"2024-05-08T22:35:15.268190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:30.819323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623032+0000","last_change":"2024-05-08T22:35:09.840351+0000","last_active":"2024-05-08T22:35:11.623032+0000","last_peered":"2024-05-08T22:35:11.623032+0000","last_clean":"2024-05-08T22:35:11.623032+0000","last_became_active":"2024-05-08T22:35:09.840000+0000","last_became_peered":"2024-05-08T22:35:09.840000+0000","last_unstale":"2024-05-08T22:35:11.623032+0000","last_undegraded":"2024-05-08T22:35:11.623032+0000","last_fullsized":"2024-05-08T22:35:11.623032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:51.428513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.846470+0000","last_change":"2024-05-08T22:35:09.389572+0000","last_active":"2024-05-08T22:35:13.846470+0000","last_peered":"2024-05-08T22:35:13.846470+0000","last_clean":"2024-05-08T22:35:13.846470+0000","last_became_active":"2024-05-08T22:35:09.389245+0000","last_became_peered":"2024-05-08T22:35:09.389245+0000","last_unstale":"2024-05-08T22:35:13.846470+0000","last_undegraded":"2024-05-08T22:35:13.846470+0000","last_fullsized":"2024-05-08T22:35:13.846470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:04.992010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.622950+0000","last_change":"2024-05-08T22:35:09.840132+0000","last_active":"2024-05-08T22:35:11.622950+0000","last_peered":"2024-05-08T22:35:11.622950+0000","last_clean":"2024-05-08T22:35:11.622950+0000","last_became_active":"2024-05-08T22:35:09.839684+0000","last_became_peered":"2024-05-08T22:35:09.839684+0000","last_unstale":"2024-05-08T22:35:11.622950+0000","last_undegraded":"2024-05-08T22:35:11.622950+0000","last_fullsized":"2024-05-08T22:35:11.622950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:32.325406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.847302+0000","last_change":"2024-05-08T22:35:09.392220+0000","last_active":"2024-05-08T22:35:13.847302+0000","last_peered":"2024-05-08T22:35:13.847302+0000","last_clean":"2024-05-08T22:35:13.847302+0000","last_became_active":"2024-05-08T22:35:09.391889+0000","last_became_peered":"2024-05-08T22:35:09.391889+0000","last_unstale":"2024-05-08T22:35:13.847302+0000","last_undegraded":"2024-05-08T22:35:13.847302+0000","last_fullsized":"2024-05-08T22:35:13.847302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:01.417894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813015+0000","last_change":"2024-05-08T22:35:09.823395+0000","last_active":"2024-05-08T22:35:10.813015+0000","last_peered":"2024-05-08T22:35:10.813015+0000","last_clean":"2024-05-08T22:35:10.813015+0000","last_became_active":"2024-05-08T22:35:09.822484+0000","last_became_peered":"2024-05-08T22:35:09.822484+0000","last_unstale":"2024-05-08T22:35:10.813015+0000","last_undegraded":"2024-05-08T22:35:10.813015+0000","last_fullsized":"2024-05-08T22:35:10.813015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:35.330789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.829259+0000","last_change":"2024-05-08T22:35:09.379904+0000","last_active":"2024-05-08T22:35:09.829259+0000","last_peered":"2024-05-08T22:35:09.829259+0000","last_clean":"2024-05-08T22:35:09.829259+0000","last_became_active":"2024-05-08T22:35:09.379533+0000","last_became_peered":"2024-05-08T22:35:09.379533+0000","last_unstale":"2024-05-08T22:35:09.829259+0000","last_undegraded":"2024-05-08T22:35:09.829259+0000","last_fullsized":"2024-05-08T22:35:09.829259+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:14.625069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.838589+0000","last_change":"2024-05-08T22:35:09.836472+0000","last_active":"2024-05-08T22:35:09.838589+0000","last_peered":"2024-05-08T22:35:09.838589+0000","last_clean":"2024-05-08T22:35:09.838589+0000","last_became_active":"2024-05-08T22:35:09.836110+0000","last_became_peered":"2024-05-08T22:35:09.836110+0000","last_unstale":"2024-05-08T22:35:09.838589+0000","last_undegraded":"2024-05-08T22:35:09.838589+0000","last_fullsized":"2024-05-08T22:35:09.838589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:48.979661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.811775+0000","last_change":"2024-05-08T22:35:09.391647+0000","last_active":"2024-05-08T22:35:09.811775+0000","last_peered":"2024-05-08T22:35:09.811775+0000","last_clean":"2024-05-08T22:35:09.811775+0000","last_became_active":"2024-05-08T22:35:09.391273+0000","last_became_peered":"2024-05-08T22:35:09.391273+0000","last_unstale":"2024-05-08T22:35:09.811775+0000","last_undegraded":"2024-05-08T22:35:09.811775+0000","last_fullsized":"2024-05-08T22:35:09.811775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:48.001313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.752901+0000","last_change":"2024-05-08T22:35:09.828713+0000","last_active":"2024-05-08T22:35:11.752901+0000","last_peered":"2024-05-08T22:35:11.752901+0000","last_clean":"2024-05-08T22:35:11.752901+0000","last_became_active":"2024-05-08T22:35:09.828370+0000","last_became_peered":"2024-05-08T22:35:09.828370+0000","last_unstale":"2024-05-08T22:35:11.752901+0000","last_undegraded":"2024-05-08T22:35:11.752901+0000","last_fullsized":"2024-05-08T22:35:11.752901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:23.328029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:16.285247+0000","last_change":"2024-05-08T22:35:09.391658+0000","last_active":"2024-05-08T22:35:16.285247+0000","last_peered":"2024-05-08T22:35:16.285247+0000","last_clean":"2024-05-08T22:35:16.285247+0000","last_became_active":"2024-05-08T22:35:09.391326+0000","last_became_peered":"2024-05-08T22:35:09.391326+0000","last_unstale":"2024-05-08T22:35:16.285247+0000","last_undegraded":"2024-05-08T22:35:16.285247+0000","last_fullsized":"2024-05-08T22:35:16.285247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:08:13.933145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604425+0000","last_change":"2024-05-08T22:35:09.837816+0000","last_active":"2024-05-08T22:35:11.604425+0000","last_peered":"2024-05-08T22:35:11.604425+0000","last_clean":"2024-05-08T22:35:11.604425+0000","last_became_active":"2024-05-08T22:35:09.837471+0000","last_became_peered":"2024-05-08T22:35:09.837471+0000","last_unstale":"2024-05-08T22:35:11.604425+0000","last_undegraded":"2024-05-08T22:35:11.604425+0000","last_fullsized":"2024-05-08T22:35:11.604425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:04.564055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813006+0000","last_change":"2024-05-08T22:35:09.827092+0000","last_active":"2024-05-08T22:35:10.813006+0000","last_peered":"2024-05-08T22:35:10.813006+0000","last_clean":"2024-05-08T22:35:10.813006+0000","last_became_active":"2024-05-08T22:35:09.826238+0000","last_became_peered":"2024-05-08T22:35:09.826238+0000","last_unstale":"2024-05-08T22:35:10.813006+0000","last_undegraded":"2024-05-08T22:35:10.813006+0000","last_fullsized":"2024-05-08T22:35:10.813006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:26:59.504555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.259760+0000","last_change":"2024-05-08T22:35:09.391555+0000","last_active":"2024-05-08T22:35:15.259760+0000","last_peered":"2024-05-08T22:35:15.259760+0000","last_clean":"2024-05-08T22:35:15.259760+0000","last_became_active":"2024-05-08T22:35:09.391294+0000","last_became_peered":"2024-05-08T22:35:09.391294+0000","last_unstale":"2024-05-08T22:35:15.259760+0000","last_undegraded":"2024-05-08T22:35:15.259760+0000","last_fullsized":"2024-05-08T22:35:15.259760+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:41.195882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825544+0000","last_change":"2024-05-08T22:35:09.822814+0000","last_active":"2024-05-08T22:35:11.825544+0000","last_peered":"2024-05-08T22:35:11.825544+0000","last_clean":"2024-05-08T22:35:11.825544+0000","last_became_active":"2024-05-08T22:35:09.822425+0000","last_became_peered":"2024-05-08T22:35:09.822425+0000","last_unstale":"2024-05-08T22:35:11.825544+0000","last_undegraded":"2024-05-08T22:35:11.825544+0000","last_fullsized":"2024-05-08T22:35:11.825544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:56.212615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.839323+0000","last_change":"2024-05-08T22:35:09.383782+0000","last_active":"2024-05-08T22:35:12.839323+0000","last_peered":"2024-05-08T22:35:12.839323+0000","last_clean":"2024-05-08T22:35:12.839323+0000","last_became_active":"2024-05-08T22:35:09.383285+0000","last_became_peered":"2024-05-08T22:35:09.383285+0000","last_unstale":"2024-05-08T22:35:12.839323+0000","last_undegraded":"2024-05-08T22:35:12.839323+0000","last_fullsized":"2024-05-08T22:35:12.839323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:42.380683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":322,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.822963+0000","last_change":"2024-05-08T22:34:20.037406+0000","last_active":"2024-05-08T22:35:09.822963+0000","last_peered":"2024-05-08T22:35:09.822963+0000","last_clean":"2024-05-08T22:35:09.822963+0000","last_became_active":"2024-05-08T22:34:20.036880+0000","last_became_peered":"2024-05-08T22:34:20.036880+0000","last_unstale":"2024-05-08T22:35:09.822963+0000","last_undegraded":"2024-05-08T22:35:09.822963+0000","last_fullsized":"2024-05-08T22:35:09.822963+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:31:54.953551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:31:54.953551+0000","last_clean_scrub_stamp":"2024-05-08T22:31:54.953551+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:00.820169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.469956+0000","last_change":"2024-05-08T22:35:09.830425+0000","last_active":"2024-05-08T22:35:10.469956+0000","last_peered":"2024-05-08T22:35:10.469956+0000","last_clean":"2024-05-08T22:35:10.469956+0000","last_became_active":"2024-05-08T22:35:09.830097+0000","last_became_peered":"2024-05-08T22:35:09.830097+0000","last_unstale":"2024-05-08T22:35:10.469956+0000","last_undegraded":"2024-05-08T22:35:10.469956+0000","last_fullsized":"2024-05-08T22:35:10.469956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:53.445798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.846273+0000","last_change":"2024-05-08T22:35:09.386717+0000","last_active":"2024-05-08T22:35:13.846273+0000","last_peered":"2024-05-08T22:35:13.846273+0000","last_clean":"2024-05-08T22:35:13.846273+0000","last_became_active":"2024-05-08T22:35:09.386371+0000","last_became_peered":"2024-05-08T22:35:09.386371+0000","last_unstale":"2024-05-08T22:35:13.846273+0000","last_undegraded":"2024-05-08T22:35:13.846273+0000","last_fullsized":"2024-05-08T22:35:13.846273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:41.061997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.816655+0000","last_change":"2024-05-08T22:35:09.394542+0000","last_active":"2024-05-08T22:35:10.816655+0000","last_peered":"2024-05-08T22:35:10.816655+0000","last_clean":"2024-05-08T22:35:10.816655+0000","last_became_active":"2024-05-08T22:35:09.394223+0000","last_became_peered":"2024-05-08T22:35:09.394223+0000","last_unstale":"2024-05-08T22:35:10.816655+0000","last_undegraded":"2024-05-08T22:35:10.816655+0000","last_fullsized":"2024-05-08T22:35:10.816655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:02.162377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470081+0000","last_change":"2024-05-08T22:35:09.825810+0000","last_active":"2024-05-08T22:35:10.470081+0000","last_peered":"2024-05-08T22:35:10.470081+0000","last_clean":"2024-05-08T22:35:10.470081+0000","last_became_active":"2024-05-08T22:35:09.825516+0000","last_became_peered":"2024-05-08T22:35:09.825516+0000","last_unstale":"2024-05-08T22:35:10.470081+0000","last_undegraded":"2024-05-08T22:35:10.470081+0000","last_fullsized":"2024-05-08T22:35:10.470081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:24.087187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.847375+0000","last_change":"2024-05-08T22:35:09.381688+0000","last_active":"2024-05-08T22:35:13.847375+0000","last_peered":"2024-05-08T22:35:13.847375+0000","last_clean":"2024-05-08T22:35:13.847375+0000","last_became_active":"2024-05-08T22:35:09.381343+0000","last_became_peered":"2024-05-08T22:35:09.381343+0000","last_unstale":"2024-05-08T22:35:13.847375+0000","last_undegraded":"2024-05-08T22:35:13.847375+0000","last_fullsized":"2024-05-08T22:35:13.847375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:14:16.142265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825676+0000","last_change":"2024-05-08T22:35:09.831689+0000","last_active":"2024-05-08T22:35:11.825676+0000","last_peered":"2024-05-08T22:35:11.825676+0000","last_clean":"2024-05-08T22:35:11.825676+0000","last_became_active":"2024-05-08T22:35:09.831337+0000","last_became_peered":"2024-05-08T22:35:09.831337+0000","last_unstale":"2024-05-08T22:35:11.825676+0000","last_undegraded":"2024-05-08T22:35:11.825676+0000","last_fullsized":"2024-05-08T22:35:11.825676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.965340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.839504+0000","last_change":"2024-05-08T22:35:09.390479+0000","last_active":"2024-05-08T22:35:12.839504+0000","last_peered":"2024-05-08T22:35:12.839504+0000","last_clean":"2024-05-08T22:35:12.839504+0000","last_became_active":"2024-05-08T22:35:09.389790+0000","last_became_peered":"2024-05-08T22:35:09.389790+0000","last_unstale":"2024-05-08T22:35:12.839504+0000","last_undegraded":"2024-05-08T22:35:12.839504+0000","last_fullsized":"2024-05-08T22:35:12.839504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:27.668801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841685+0000","last_change":"2024-05-08T22:35:09.836123+0000","last_active":"2024-05-08T22:35:11.841685+0000","last_peered":"2024-05-08T22:35:11.841685+0000","last_clean":"2024-05-08T22:35:11.841685+0000","last_became_active":"2024-05-08T22:35:09.835840+0000","last_became_peered":"2024-05-08T22:35:09.835840+0000","last_unstale":"2024-05-08T22:35:11.841685+0000","last_undegraded":"2024-05-08T22:35:11.841685+0000","last_fullsized":"2024-05-08T22:35:11.841685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:52.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.840918+0000","last_change":"2024-05-08T22:35:09.394658+0000","last_active":"2024-05-08T22:35:12.840918+0000","last_peered":"2024-05-08T22:35:12.840918+0000","last_clean":"2024-05-08T22:35:12.840918+0000","last_became_active":"2024-05-08T22:35:09.394351+0000","last_became_peered":"2024-05-08T22:35:09.394351+0000","last_unstale":"2024-05-08T22:35:12.840918+0000","last_undegraded":"2024-05-08T22:35:12.840918+0000","last_fullsized":"2024-05-08T22:35:12.840918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:32.954612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.834139+0000","last_change":"2024-05-08T22:35:09.830853+0000","last_active":"2024-05-08T22:35:09.834139+0000","last_peered":"2024-05-08T22:35:09.834139+0000","last_clean":"2024-05-08T22:35:09.834139+0000","last_became_active":"2024-05-08T22:35:09.830471+0000","last_became_peered":"2024-05-08T22:35:09.830471+0000","last_unstale":"2024-05-08T22:35:09.834139+0000","last_undegraded":"2024-05-08T22:35:09.834139+0000","last_fullsized":"2024-05-08T22:35:09.834139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:13.960966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.265416+0000","last_change":"2024-05-08T22:35:09.382515+0000","last_active":"2024-05-08T22:35:15.265416+0000","last_peered":"2024-05-08T22:35:15.265416+0000","last_clean":"2024-05-08T22:35:15.265416+0000","last_became_active":"2024-05-08T22:35:09.382168+0000","last_became_peered":"2024-05-08T22:35:09.382168+0000","last_unstale":"2024-05-08T22:35:15.265416+0000","last_undegraded":"2024-05-08T22:35:15.265416+0000","last_fullsized":"2024-05-08T22:35:15.265416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:10.248542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.837958+0000","last_change":"2024-05-08T22:35:09.835925+0000","last_active":"2024-05-08T22:35:09.837958+0000","last_peered":"2024-05-08T22:35:09.837958+0000","last_clean":"2024-05-08T22:35:09.837958+0000","last_became_active":"2024-05-08T22:35:09.835547+0000","last_became_peered":"2024-05-08T22:35:09.835547+0000","last_unstale":"2024-05-08T22:35:09.837958+0000","last_undegraded":"2024-05-08T22:35:09.837958+0000","last_fullsized":"2024-05-08T22:35:09.837958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:36.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841302+0000","last_change":"2024-05-08T22:35:09.385035+0000","last_active":"2024-05-08T22:35:11.841302+0000","last_peered":"2024-05-08T22:35:11.841302+0000","last_clean":"2024-05-08T22:35:11.841302+0000","last_became_active":"2024-05-08T22:35:09.384274+0000","last_became_peered":"2024-05-08T22:35:09.384274+0000","last_unstale":"2024-05-08T22:35:11.841302+0000","last_undegraded":"2024-05-08T22:35:11.841302+0000","last_fullsized":"2024-05-08T22:35:11.841302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:50.654985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825567+0000","last_change":"2024-05-08T22:35:09.821454+0000","last_active":"2024-05-08T22:35:11.825567+0000","last_peered":"2024-05-08T22:35:11.825567+0000","last_clean":"2024-05-08T22:35:11.825567+0000","last_became_active":"2024-05-08T22:35:09.821199+0000","last_became_peered":"2024-05-08T22:35:09.821199+0000","last_unstale":"2024-05-08T22:35:11.825567+0000","last_undegraded":"2024-05-08T22:35:11.825567+0000","last_fullsized":"2024-05-08T22:35:11.825567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:02.677307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.840347+0000","last_change":"2024-05-08T22:35:09.394027+0000","last_active":"2024-05-08T22:35:12.840347+0000","last_peered":"2024-05-08T22:35:12.840347+0000","last_clean":"2024-05-08T22:35:12.840347+0000","last_became_active":"2024-05-08T22:35:09.393622+0000","last_became_peered":"2024-05-08T22:35:09.393622+0000","last_unstale":"2024-05-08T22:35:12.840347+0000","last_undegraded":"2024-05-08T22:35:12.840347+0000","last_fullsized":"2024-05-08T22:35:12.840347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:13.415635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623012+0000","last_change":"2024-05-08T22:35:09.842928+0000","last_active":"2024-05-08T22:35:11.623012+0000","last_peered":"2024-05-08T22:35:11.623012+0000","last_clean":"2024-05-08T22:35:11.623012+0000","last_became_active":"2024-05-08T22:35:09.842576+0000","last_became_peered":"2024-05-08T22:35:09.842576+0000","last_unstale":"2024-05-08T22:35:11.623012+0000","last_undegraded":"2024-05-08T22:35:11.623012+0000","last_fullsized":"2024-05-08T22:35:11.623012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:03.526259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.846645+0000","last_change":"2024-05-08T22:35:09.396325+0000","last_active":"2024-05-08T22:35:13.846645+0000","last_peered":"2024-05-08T22:35:13.846645+0000","last_clean":"2024-05-08T22:35:13.846645+0000","last_became_active":"2024-05-08T22:35:09.395656+0000","last_became_peered":"2024-05-08T22:35:09.395656+0000","last_unstale":"2024-05-08T22:35:13.846645+0000","last_undegraded":"2024-05-08T22:35:13.846645+0000","last_fullsized":"2024-05-08T22:35:13.846645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:30.326694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623076+0000","last_change":"2024-05-08T22:35:09.840682+0000","last_active":"2024-05-08T22:35:11.623076+0000","last_peered":"2024-05-08T22:35:11.623076+0000","last_clean":"2024-05-08T22:35:11.623076+0000","last_became_active":"2024-05-08T22:35:09.840315+0000","last_became_peered":"2024-05-08T22:35:09.840315+0000","last_unstale":"2024-05-08T22:35:11.623076+0000","last_undegraded":"2024-05-08T22:35:11.623076+0000","last_fullsized":"2024-05-08T22:35:11.623076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:13.884569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.257932+0000","last_change":"2024-05-08T22:35:09.389169+0000","last_active":"2024-05-08T22:35:15.257932+0000","last_peered":"2024-05-08T22:35:15.257932+0000","last_clean":"2024-05-08T22:35:15.257932+0000","last_became_active":"2024-05-08T22:35:09.388836+0000","last_became_peered":"2024-05-08T22:35:09.388836+0000","last_unstale":"2024-05-08T22:35:15.257932+0000","last_undegraded":"2024-05-08T22:35:15.257932+0000","last_fullsized":"2024-05-08T22:35:15.257932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:08.960719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841499+0000","last_change":"2024-05-08T22:35:09.833885+0000","last_active":"2024-05-08T22:35:11.841499+0000","last_peered":"2024-05-08T22:35:11.841499+0000","last_clean":"2024-05-08T22:35:11.841499+0000","last_became_active":"2024-05-08T22:35:09.833596+0000","last_became_peered":"2024-05-08T22:35:09.833596+0000","last_unstale":"2024-05-08T22:35:11.841499+0000","last_undegraded":"2024-05-08T22:35:11.841499+0000","last_fullsized":"2024-05-08T22:35:11.841499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:29.673837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267349+0000","last_change":"2024-05-08T22:35:09.389023+0000","last_active":"2024-05-08T22:35:15.267349+0000","last_peered":"2024-05-08T22:35:15.267349+0000","last_clean":"2024-05-08T22:35:15.267349+0000","last_became_active":"2024-05-08T22:35:09.388721+0000","last_became_peered":"2024-05-08T22:35:09.388721+0000","last_unstale":"2024-05-08T22:35:15.267349+0000","last_undegraded":"2024-05-08T22:35:15.267349+0000","last_fullsized":"2024-05-08T22:35:15.267349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:41.776592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623109+0000","last_change":"2024-05-08T22:35:09.839499+0000","last_active":"2024-05-08T22:35:11.623109+0000","last_peered":"2024-05-08T22:35:11.623109+0000","last_clean":"2024-05-08T22:35:11.623109+0000","last_became_active":"2024-05-08T22:35:09.839146+0000","last_became_peered":"2024-05-08T22:35:09.839146+0000","last_unstale":"2024-05-08T22:35:11.623109+0000","last_undegraded":"2024-05-08T22:35:11.623109+0000","last_fullsized":"2024-05-08T22:35:11.623109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:45.636827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.839015+0000","last_change":"2024-05-08T22:35:09.836710+0000","last_active":"2024-05-08T22:35:09.839015+0000","last_peered":"2024-05-08T22:35:09.839015+0000","last_clean":"2024-05-08T22:35:09.839015+0000","last_became_active":"2024-05-08T22:35:09.835838+0000","last_became_peered":"2024-05-08T22:35:09.835838+0000","last_unstale":"2024-05-08T22:35:09.839015+0000","last_undegraded":"2024-05-08T22:35:09.839015+0000","last_fullsized":"2024-05-08T22:35:09.839015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:00.567805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841269+0000","last_change":"2024-05-08T22:35:09.830148+0000","last_active":"2024-05-08T22:35:11.841269+0000","last_peered":"2024-05-08T22:35:11.841269+0000","last_clean":"2024-05-08T22:35:11.841269+0000","last_became_active":"2024-05-08T22:35:09.829820+0000","last_became_peered":"2024-05-08T22:35:09.829820+0000","last_unstale":"2024-05-08T22:35:11.841269+0000","last_undegraded":"2024-05-08T22:35:11.841269+0000","last_fullsized":"2024-05-08T22:35:11.841269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:38.840993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841227+0000","last_change":"2024-05-08T22:35:09.830428+0000","last_active":"2024-05-08T22:35:11.841227+0000","last_peered":"2024-05-08T22:35:11.841227+0000","last_clean":"2024-05-08T22:35:11.841227+0000","last_became_active":"2024-05-08T22:35:09.829736+0000","last_became_peered":"2024-05-08T22:35:09.829736+0000","last_unstale":"2024-05-08T22:35:11.841227+0000","last_undegraded":"2024-05-08T22:35:11.841227+0000","last_fullsized":"2024-05-08T22:35:11.841227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:54.458662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841649+0000","last_change":"2024-05-08T22:35:09.836200+0000","last_active":"2024-05-08T22:35:11.841649+0000","last_peered":"2024-05-08T22:35:11.841649+0000","last_clean":"2024-05-08T22:35:11.841649+0000","last_became_active":"2024-05-08T22:35:09.835892+0000","last_became_peered":"2024-05-08T22:35:09.835892+0000","last_unstale":"2024-05-08T22:35:11.841649+0000","last_undegraded":"2024-05-08T22:35:11.841649+0000","last_fullsized":"2024-05-08T22:35:11.841649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:42.487982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470102+0000","last_change":"2024-05-08T22:35:09.834026+0000","last_active":"2024-05-08T22:35:10.470102+0000","last_peered":"2024-05-08T22:35:10.470102+0000","last_clean":"2024-05-08T22:35:10.470102+0000","last_became_active":"2024-05-08T22:35:09.833659+0000","last_became_peered":"2024-05-08T22:35:09.833659+0000","last_unstale":"2024-05-08T22:35:10.470102+0000","last_undegraded":"2024-05-08T22:35:10.470102+0000","last_fullsized":"2024-05-08T22:35:10.470102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:00.536853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.839061+0000","last_change":"2024-05-08T22:35:09.836928+0000","last_active":"2024-05-08T22:35:09.839061+0000","last_peered":"2024-05-08T22:35:09.839061+0000","last_clean":"2024-05-08T22:35:09.839061+0000","last_became_active":"2024-05-08T22:35:09.836629+0000","last_became_peered":"2024-05-08T22:35:09.836629+0000","last_unstale":"2024-05-08T22:35:09.839061+0000","last_undegraded":"2024-05-08T22:35:09.839061+0000","last_fullsized":"2024-05-08T22:35:09.839061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:43.475121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841544+0000","last_change":"2024-05-08T22:35:09.837685+0000","last_active":"2024-05-08T22:35:11.841544+0000","last_peered":"2024-05-08T22:35:11.841544+0000","last_clean":"2024-05-08T22:35:11.841544+0000","last_became_active":"2024-05-08T22:35:09.837409+0000","last_became_peered":"2024-05-08T22:35:09.837409+0000","last_unstale":"2024-05-08T22:35:11.841544+0000","last_undegraded":"2024-05-08T22:35:11.841544+0000","last_fullsized":"2024-05-08T22:35:11.841544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:32.076361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825653+0000","last_change":"2024-05-08T22:35:09.833099+0000","last_active":"2024-05-08T22:35:11.825653+0000","last_peered":"2024-05-08T22:35:11.825653+0000","last_clean":"2024-05-08T22:35:11.825653+0000","last_became_active":"2024-05-08T22:35:09.832814+0000","last_became_peered":"2024-05-08T22:35:09.832814+0000","last_unstale":"2024-05-08T22:35:11.825653+0000","last_undegraded":"2024-05-08T22:35:11.825653+0000","last_fullsized":"2024-05-08T22:35:11.825653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:40.116060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812821+0000","last_change":"2024-05-08T22:35:09.818507+0000","last_active":"2024-05-08T22:35:10.812821+0000","last_peered":"2024-05-08T22:35:10.812821+0000","last_clean":"2024-05-08T22:35:10.812821+0000","last_became_active":"2024-05-08T22:35:09.818139+0000","last_became_peered":"2024-05-08T22:35:09.818139+0000","last_unstale":"2024-05-08T22:35:10.812821+0000","last_undegraded":"2024-05-08T22:35:10.812821+0000","last_fullsized":"2024-05-08T22:35:10.812821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:33.747011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825503+0000","last_change":"2024-05-08T22:35:09.832232+0000","last_active":"2024-05-08T22:35:11.825503+0000","last_peered":"2024-05-08T22:35:11.825503+0000","last_clean":"2024-05-08T22:35:11.825503+0000","last_became_active":"2024-05-08T22:35:09.831889+0000","last_became_peered":"2024-05-08T22:35:09.831889+0000","last_unstale":"2024-05-08T22:35:11.825503+0000","last_undegraded":"2024-05-08T22:35:11.825503+0000","last_fullsized":"2024-05-08T22:35:11.825503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:01.802009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812645+0000","last_change":"2024-05-08T22:35:09.838051+0000","last_active":"2024-05-08T22:35:10.812645+0000","last_peered":"2024-05-08T22:35:10.812645+0000","last_clean":"2024-05-08T22:35:10.812645+0000","last_became_active":"2024-05-08T22:35:09.837429+0000","last_became_peered":"2024-05-08T22:35:09.837429+0000","last_unstale":"2024-05-08T22:35:10.812645+0000","last_undegraded":"2024-05-08T22:35:10.812645+0000","last_fullsized":"2024-05-08T22:35:10.812645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.191374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470060+0000","last_change":"2024-05-08T22:35:09.830834+0000","last_active":"2024-05-08T22:35:10.470060+0000","last_peered":"2024-05-08T22:35:10.470060+0000","last_clean":"2024-05-08T22:35:10.470060+0000","last_became_active":"2024-05-08T22:35:09.830475+0000","last_became_peered":"2024-05-08T22:35:09.830475+0000","last_unstale":"2024-05-08T22:35:10.470060+0000","last_undegraded":"2024-05-08T22:35:10.470060+0000","last_fullsized":"2024-05-08T22:35:10.470060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:06:10.996668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.838544+0000","last_change":"2024-05-08T22:35:09.836496+0000","last_active":"2024-05-08T22:35:09.838544+0000","last_peered":"2024-05-08T22:35:09.838544+0000","last_clean":"2024-05-08T22:35:09.838544+0000","last_became_active":"2024-05-08T22:35:09.835787+0000","last_became_peered":"2024-05-08T22:35:09.835787+0000","last_unstale":"2024-05-08T22:35:09.838544+0000","last_undegraded":"2024-05-08T22:35:09.838544+0000","last_fullsized":"2024-05-08T22:35:09.838544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:30.395286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43378,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":24,"num_read_kb":37,"num_write":169,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":94426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":856064,"data_stored":410021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]}]},{"osd":10,"up_from":67,"seq":287762808847,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32596,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712556,"statfs":{"total":95995035648,"available":95961657344,"internally_reserved":0,"allocated":1519616,"data_stored":1022487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]}]},{"osd":9,"up_from":62,"seq":266287972370,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":872448,"data_stored":410257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.45700000000000002}]},{"osd":4,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53800000000000003}]},{"osd":5,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.55700000000000005}]},{"osd":6,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.496}]},{"osd":7,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.70499999999999996}]},{"osd":8,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.46700000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":8,"up_from":55,"seq":236223201301,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":868352,"data_stored":409896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.438}]},{"osd":1,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61399999999999999}]},{"osd":3,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":5.468,"5min":5.468,"15min":5.468},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.79800000000000004}]},{"osd":5,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.71099999999999997}]},{"osd":6,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":7.3689999999999998,"5min":7.3689999999999998,"15min":7.3689999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.5620000000000003,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.68999999999999995}]},{"osd":7,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.88600000000000001}]},{"osd":9,"last update":"Wed May 8 22:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.751}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":7,"up_from":48,"seq":206158430233,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32552,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712600,"statfs":{"total":95995035648,"available":95961702400,"internally_reserved":0,"allocated":1474560,"data_stored":1003260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40100000000000002}]},{"osd":1,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.78000000000000003}]},{"osd":3,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.65000000000000002}]},{"osd":4,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.755}]},{"osd":5,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.82999999999999996}]},{"osd":6,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.84699999999999998}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.89800000000000002}]},{"osd":9,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.93899999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":6,"up_from":43,"seq":184683593756,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":409551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39000000000000001,"15min":0.38800000000000001},"min":{"1min":0.29399999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42499999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.76500000000000001}]},{"osd":1,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.82399999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77000000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.754},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.86299999999999999}]},{"osd":3,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47699999999999998,"15min":0.48599999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56999999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.49299999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.72899999999999998}]},{"osd":4,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63},"min":{"1min":0.44800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77500000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.874,"15min":0.874},"last":0.66500000000000004}]},{"osd":5,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.48199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":1.0369999999999999}]},{"osd":7,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.71299999999999997}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":1.0149999999999999}]},{"osd":9,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.95499999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36076,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709076,"statfs":{"total":95995035648,"available":95958093824,"internally_reserved":0,"allocated":888832,"data_stored":431719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48399999999999999,"15min":0.47199999999999998},"min":{"1min":0.317,"5min":0.311,"15min":0.311},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46300000000000002}]},{"osd":2,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83599999999999997,"5min":1.627,"15min":1.627},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.753}]},{"osd":3,"last update":"Wed May 8 22:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.35299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.621},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.40600000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.89300000000000002,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.53700000000000003}]},{"osd":4,"last update":"Wed May 8 22:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42499999999999999,"15min":0.41799999999999998},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.45100000000000001,"15min":0.435},"min":{"1min":0.32600000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.55700000000000005}]},{"osd":5,"last update":"Wed May 8 22:34:52 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.83399999999999996}]},{"osd":6,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.47599999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66600000000000004}]},{"osd":8,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.80300000000000005}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65100000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738418,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":864256,"data_stored":410133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.40600000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.52100000000000002,"15min":0.499},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.58599999999999997}]},{"osd":2,"last update":"Wed May 8 22:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.40200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51400000000000001,"15min":0.48799999999999999},"min":{"1min":0.36099999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.76500000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.56599999999999995}]},{"osd":3,"last update":"Wed May 8 22:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.39600000000000002,"15min":0.38700000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.42699999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45300000000000001}]},{"osd":5,"last update":"Wed May 8 22:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70599999999999996,"5min":0.751,"15min":0.751},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90000000000000002,"5min":1.085,"15min":1.085},"last":0.52600000000000002}]},{"osd":6,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78700000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83899999999999997,"5min":0.872,"15min":0.872},"last":0.377}]},{"osd":7,"last update":"Wed May 8 22:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53500000000000003}]},{"osd":8,"last update":"Wed May 8 22:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.433}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.443},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.48199999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411370,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":872448,"data_stored":410942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.502},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71799999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.72499999999999998}]},{"osd":1,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.40500000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.35799999999999998}]},{"osd":3,"last update":"Wed May 8 22:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.496},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73399999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed May 8 22:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.44},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.33200000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64100000000000001,"5min":0.748,"15min":0.748},"last":0.33400000000000002}]},{"osd":6,"last update":"Wed May 8 22:35:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.50600000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed May 8 22:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.47299999999999998}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.74399999999999999}]},{"osd":9,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.627},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.66900000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]}]},{"osd":3,"up_from":24,"seq":103079215143,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36072,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709080,"statfs":{"total":95995035648,"available":95958097920,"internally_reserved":0,"allocated":884736,"data_stored":409986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.33600000000000002},"min":{"1min":0.27200000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.54100000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.318},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.36399999999999999,"15min":0.35199999999999998},"min":{"1min":0.28899999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58899999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.51400000000000001}]},{"osd":1,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72999999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54100000000000004,"15min":0.52800000000000002},"min":{"1min":0.38800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.41799999999999998}]},{"osd":2,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88100000000000001,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.50700000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.47599999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53200000000000003}]},{"osd":5,"last update":"Wed May 8 22:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85499999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.48499999999999999}]},{"osd":6,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.435}]},{"osd":7,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56399999999999995}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.496},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.53900000000000003}]},{"osd":9,"last update":"Wed May 8 22:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.67700000000000005}]},{"osd":10,"last update":"Wed May 8 22:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.501},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.58899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":864256,"data_stored":410035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.38100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.42699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.39900000000000002},"min":{"1min":0.27000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77700000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.26400000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.63800000000000001,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75700000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.77200000000000002}]},{"osd":3,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.35499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89200000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.79700000000000004}]},{"osd":6,"last update":"Wed May 8 22:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.40699999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.747},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.82499999999999996}]},{"osd":7,"last update":"Wed May 8 22:34:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59399999999999997}]},{"osd":8,"last update":"Wed May 8 22:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.997}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.95199999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1490944,"data_stored":1002758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.50800000000000001,"15min":0.499},"min":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.623},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.35899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51800000000000002}]},{"osd":1,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.65400000000000003}]},{"osd":2,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.379,"15min":0.372},"min":{"1min":0.28199999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.374}]},{"osd":3,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed May 8 22:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40899999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68999999999999995}]},{"osd":7,"last update":"Wed May 8 22:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76200000000000001}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39400000000000002}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.72699999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":39851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":40228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:35:18.112 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T22:35:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:17 smithi079 ceph-mon[46738]: pgmap v225: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 154 B/s rd, 6.0 KiB/s wr, 15 op/s 2024-05-08T22:35:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:17 smithi079 ceph-mon[46738]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:17 smithi079 ceph-mon[46738]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:17 smithi079 ceph-mon[46738]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:17 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/850508515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:17 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1068461851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:18.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.272+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1524965544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9080ffd80 msgr2=0x7ff9081001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.272+0000 7ff90cc6f700 1 --2- 172.21.15.62:0/1524965544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9080ffd80 0x7ff9081001a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00099c0 tx=0x7ff8f0009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.272+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1524965544 shutdown_connections 2024-05-08T22:35:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.272+0000 7ff90cc6f700 1 --2- 172.21.15.62:0/1524965544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff908101910 0x7ff9081081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.272+0000 7ff90cc6f700 1 --2- 172.21.15.62:0/1524965544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9080ffd80 0x7ff9081001a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.272+0000 7ff90cc6f700 1 --2- 172.21.15.62:0/1524965544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff908100f70 0x7ff9081013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.272+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1524965544 >> 172.21.15.62:0/1524965544 conn(0x7ff9080fb2f0 msgr2=0x7ff9080fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.272+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1524965544 shutdown_connections 2024-05-08T22:35:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.272+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1524965544 wait complete. 2024-05-08T22:35:18.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff90cc6f700 1 Processor -- start 2024-05-08T22:35:18.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff90cc6f700 1 -- start start 2024-05-08T22:35:18.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff90cc6f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9080ffd80 0x7ff908069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff90cc6f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff908100f70 0x7ff90819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff90cc6f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff908101910 0x7ff90819e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff90cc6f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90819e8c0 con 0x7ff908100f70 2024-05-08T22:35:18.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff90cc6f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90819ea30 con 0x7ff908101910 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff90cc6f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90819ebf0 con 0x7ff9080ffd80 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff905d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff908100f70 0x7ff90819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff905d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff908100f70 0x7ff90819bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60500/0 (socket says 172.21.15.62:60500) 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff905d9b700 1 -- 172.21.15.62:0/1453443108 learned_addr learned my addr 172.21.15.62:0/1453443108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff90659c700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9080ffd80 0x7ff908069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.273+0000 7ff906d9d700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff908101910 0x7ff90819e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.274+0000 7ff905d9b700 1 -- 172.21.15.62:0/1453443108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9080ffd80 msgr2=0x7ff908069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.274+0000 7ff905d9b700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9080ffd80 0x7ff908069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.274+0000 7ff905d9b700 1 -- 172.21.15.62:0/1453443108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff908101910 msgr2=0x7ff90819e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.274+0000 7ff905d9b700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff908101910 0x7ff90819e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.274+0000 7ff905d9b700 1 -- 172.21.15.62:0/1453443108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f00095f0 con 0x7ff908100f70 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.274+0000 7ff905d9b700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff908100f70 0x7ff90819bad0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800d950 tx=0x7ff8f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.274+0000 7ff8ff7fe700 1 -- 172.21.15.62:0/1453443108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f800f890 con 0x7ff908100f70 2024-05-08T22:35:18.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.274+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1453443108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90819eed0 con 0x7ff908100f70 2024-05-08T22:35:18.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.274+0000 7ff8ff7fe700 1 -- 172.21.15.62:0/1453443108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8f8010460 con 0x7ff908100f70 2024-05-08T22:35:18.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.274+0000 7ff8ff7fe700 1 -- 172.21.15.62:0/1453443108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f800e610 con 0x7ff908100f70 2024-05-08T22:35:18.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.274+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1453443108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90819f350 con 0x7ff908100f70 2024-05-08T22:35:18.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.275+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1453443108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff908107220 con 0x7ff908100f70 2024-05-08T22:35:18.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.276+0000 7ff8ff7fe700 1 -- 172.21.15.62:0/1453443108 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff8f80108d0 con 0x7ff908100f70 2024-05-08T22:35:18.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.276+0000 7ff8ff7fe700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff8f40a11b0 0x7ff8f40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:18.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.277+0000 7ff90659c700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff8f40a11b0 0x7ff8f40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:18.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.277+0000 7ff8ff7fe700 1 -- 172.21.15.62:0/1453443108 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff8f80cc670 con 0x7ff908100f70 2024-05-08T22:35:18.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.277+0000 7ff90659c700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff8f40a11b0 0x7ff8f40a3670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0003820 tx=0x7ff8f0005920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:18.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.281+0000 7ff8ff7fe700 1 -- 172.21.15.62:0/1453443108 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff8f8095e50 con 0x7ff908100f70 2024-05-08T22:35:18.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:17 smithi066 ceph-mon[46830]: pgmap v225: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 154 B/s rd, 6.0 KiB/s wr, 15 op/s 2024-05-08T22:35:18.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:17 smithi066 ceph-mon[46830]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:18.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:17 smithi066 ceph-mon[46830]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:18.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:17 smithi066 ceph-mon[46830]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:18.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:17 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/850508515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:18.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:17 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1068461851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:18.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.485+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1453443108 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff90802ceb0 con 0x7ff8f40a11b0 2024-05-08T22:35:18.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.495+0000 7ff8ff7fe700 1 -- 172.21.15.62:0/1453443108 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+328600 (secure 0 0 0) 0x7ff90802ceb0 con 0x7ff8f40a11b0 2024-05-08T22:35:18.498 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1453443108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff8f40a11b0 msgr2=0x7ff8f40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff8f40a11b0 0x7ff8f40a3670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0003820 tx=0x7ff8f0005920 comp rx=0 tx=0).stop 2024-05-08T22:35:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1453443108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff908100f70 msgr2=0x7ff90819bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff908100f70 0x7ff90819bad0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800d950 tx=0x7ff8f800dc60 comp rx=0 tx=0).stop 2024-05-08T22:35:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1453443108 shutdown_connections 2024-05-08T22:35:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff8f40a11b0 0x7ff8f40a3670 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff908101910 0x7ff90819e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9080ffd80 0x7ff908069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 --2- 172.21.15.62:0/1453443108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff908100f70 0x7ff90819bad0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1453443108 >> 172.21.15.62:0/1453443108 conn(0x7ff9080fb2f0 msgr2=0x7ff908104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1453443108 shutdown_connections 2024-05-08T22:35:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:18.498+0000 7ff90cc6f700 1 -- 172.21.15.62:0/1453443108 wait complete. 2024-05-08T22:35:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-05-08T22:35:18.510 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-05-08T22:35:18.010888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633980,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":150,"num_read_kb":146,"num_write":352,"num_write_kb":3034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":288,"ondisk_log_size":288,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381288,"kb_used_data":12028,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124560536,"statfs":{"total":1151940427776,"available":1151549988864,"internally_reserved":0,"allocated":12316672,"data_stored":6741045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":377992518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43612,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":24,"num_read_kb":37,"num_write":171,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.203332"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.840865+0000","last_change":"2024-05-08T22:35:09.840453+0000","last_active":"2024-05-08T22:35:09.840865+0000","last_peered":"2024-05-08T22:35:09.840865+0000","last_clean":"2024-05-08T22:35:09.840865+0000","last_became_active":"2024-05-08T22:35:09.840093+0000","last_became_peered":"2024-05-08T22:35:09.840093+0000","last_unstale":"2024-05-08T22:35:09.840865+0000","last_undegraded":"2024-05-08T22:35:09.840865+0000","last_fullsized":"2024-05-08T22:35:09.840865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:13.265286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840996+0000","last_change":"2024-05-08T22:35:09.832735+0000","last_active":"2024-05-08T22:35:11.840996+0000","last_peered":"2024-05-08T22:35:11.840996+0000","last_clean":"2024-05-08T22:35:11.840996+0000","last_became_active":"2024-05-08T22:35:09.832419+0000","last_became_peered":"2024-05-08T22:35:09.832419+0000","last_unstale":"2024-05-08T22:35:11.840996+0000","last_undegraded":"2024-05-08T22:35:11.840996+0000","last_fullsized":"2024-05-08T22:35:11.840996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:19.711398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841216+0000","last_change":"2024-05-08T22:35:09.835604+0000","last_active":"2024-05-08T22:35:11.841216+0000","last_peered":"2024-05-08T22:35:11.841216+0000","last_clean":"2024-05-08T22:35:11.841216+0000","last_became_active":"2024-05-08T22:35:09.835235+0000","last_became_peered":"2024-05-08T22:35:09.835235+0000","last_unstale":"2024-05-08T22:35:11.841216+0000","last_undegraded":"2024-05-08T22:35:11.841216+0000","last_fullsized":"2024-05-08T22:35:11.841216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:35:10.722947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.831424+0000","last_change":"2024-05-08T22:35:09.824876+0000","last_active":"2024-05-08T22:35:09.831424+0000","last_peered":"2024-05-08T22:35:09.831424+0000","last_clean":"2024-05-08T22:35:09.831424+0000","last_became_active":"2024-05-08T22:35:09.823494+0000","last_became_peered":"2024-05-08T22:35:09.823494+0000","last_unstale":"2024-05-08T22:35:09.831424+0000","last_undegraded":"2024-05-08T22:35:09.831424+0000","last_fullsized":"2024-05-08T22:35:09.831424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:20.099947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.828965+0000","last_change":"2024-05-08T22:35:09.825647+0000","last_active":"2024-05-08T22:35:09.828965+0000","last_peered":"2024-05-08T22:35:09.828965+0000","last_clean":"2024-05-08T22:35:09.828965+0000","last_became_active":"2024-05-08T22:35:09.825322+0000","last_became_peered":"2024-05-08T22:35:09.825322+0000","last_unstale":"2024-05-08T22:35:09.828965+0000","last_undegraded":"2024-05-08T22:35:09.828965+0000","last_fullsized":"2024-05-08T22:35:09.828965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:29.080541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604425+0000","last_change":"2024-05-08T22:35:09.841894+0000","last_active":"2024-05-08T22:35:11.604425+0000","last_peered":"2024-05-08T22:35:11.604425+0000","last_clean":"2024-05-08T22:35:11.604425+0000","last_became_active":"2024-05-08T22:35:09.841429+0000","last_became_peered":"2024-05-08T22:35:09.841429+0000","last_unstale":"2024-05-08T22:35:11.604425+0000","last_undegraded":"2024-05-08T22:35:11.604425+0000","last_fullsized":"2024-05-08T22:35:11.604425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:50.859186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.622990+0000","last_change":"2024-05-08T22:35:09.834010+0000","last_active":"2024-05-08T22:35:11.622990+0000","last_peered":"2024-05-08T22:35:11.622990+0000","last_clean":"2024-05-08T22:35:11.622990+0000","last_became_active":"2024-05-08T22:35:09.833633+0000","last_became_peered":"2024-05-08T22:35:09.833633+0000","last_unstale":"2024-05-08T22:35:11.622990+0000","last_undegraded":"2024-05-08T22:35:11.622990+0000","last_fullsized":"2024-05-08T22:35:11.622990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:28.854383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825479+0000","last_change":"2024-05-08T22:35:09.820817+0000","last_active":"2024-05-08T22:35:11.825479+0000","last_peered":"2024-05-08T22:35:11.825479+0000","last_clean":"2024-05-08T22:35:11.825479+0000","last_became_active":"2024-05-08T22:35:09.820521+0000","last_became_peered":"2024-05-08T22:35:09.820521+0000","last_unstale":"2024-05-08T22:35:11.825479+0000","last_undegraded":"2024-05-08T22:35:11.825479+0000","last_fullsized":"2024-05-08T22:35:11.825479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:17.312794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812985+0000","last_change":"2024-05-08T22:35:09.838821+0000","last_active":"2024-05-08T22:35:10.812985+0000","last_peered":"2024-05-08T22:35:10.812985+0000","last_clean":"2024-05-08T22:35:10.812985+0000","last_became_active":"2024-05-08T22:35:09.836588+0000","last_became_peered":"2024-05-08T22:35:09.836588+0000","last_unstale":"2024-05-08T22:35:10.812985+0000","last_undegraded":"2024-05-08T22:35:10.812985+0000","last_fullsized":"2024-05-08T22:35:10.812985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:52.884827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604696+0000","last_change":"2024-05-08T22:35:09.836222+0000","last_active":"2024-05-08T22:35:11.604696+0000","last_peered":"2024-05-08T22:35:11.604696+0000","last_clean":"2024-05-08T22:35:11.604696+0000","last_became_active":"2024-05-08T22:35:09.835810+0000","last_became_peered":"2024-05-08T22:35:09.835810+0000","last_unstale":"2024-05-08T22:35:11.604696+0000","last_undegraded":"2024-05-08T22:35:11.604696+0000","last_fullsized":"2024-05-08T22:35:11.604696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:38:14.075576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623061+0000","last_change":"2024-05-08T22:35:09.835422+0000","last_active":"2024-05-08T22:35:11.623061+0000","last_peered":"2024-05-08T22:35:11.623061+0000","last_clean":"2024-05-08T22:35:11.623061+0000","last_became_active":"2024-05-08T22:35:09.835100+0000","last_became_peered":"2024-05-08T22:35:09.835100+0000","last_unstale":"2024-05-08T22:35:11.623061+0000","last_undegraded":"2024-05-08T22:35:11.623061+0000","last_fullsized":"2024-05-08T22:35:11.623061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:12.099286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.832873+0000","last_change":"2024-05-08T22:35:09.827007+0000","last_active":"2024-05-08T22:35:09.832873+0000","last_peered":"2024-05-08T22:35:09.832873+0000","last_clean":"2024-05-08T22:35:09.832873+0000","last_became_active":"2024-05-08T22:35:09.826639+0000","last_became_peered":"2024-05-08T22:35:09.826639+0000","last_unstale":"2024-05-08T22:35:09.832873+0000","last_undegraded":"2024-05-08T22:35:09.832873+0000","last_fullsized":"2024-05-08T22:35:09.832873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:39.999049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470041+0000","last_change":"2024-05-08T22:35:09.828507+0000","last_active":"2024-05-08T22:35:10.470041+0000","last_peered":"2024-05-08T22:35:10.470041+0000","last_clean":"2024-05-08T22:35:10.470041+0000","last_became_active":"2024-05-08T22:35:09.828076+0000","last_became_peered":"2024-05-08T22:35:09.828076+0000","last_unstale":"2024-05-08T22:35:10.470041+0000","last_undegraded":"2024-05-08T22:35:10.470041+0000","last_fullsized":"2024-05-08T22:35:10.470041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:33.475538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840770+0000","last_change":"2024-05-08T22:35:09.820506+0000","last_active":"2024-05-08T22:35:11.840770+0000","last_peered":"2024-05-08T22:35:11.840770+0000","last_clean":"2024-05-08T22:35:11.840770+0000","last_became_active":"2024-05-08T22:35:09.820208+0000","last_became_peered":"2024-05-08T22:35:09.820208+0000","last_unstale":"2024-05-08T22:35:11.840770+0000","last_undegraded":"2024-05-08T22:35:11.840770+0000","last_fullsized":"2024-05-08T22:35:11.840770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:47:00.753509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840848+0000","last_change":"2024-05-08T22:35:09.837171+0000","last_active":"2024-05-08T22:35:11.840848+0000","last_peered":"2024-05-08T22:35:11.840848+0000","last_clean":"2024-05-08T22:35:11.840848+0000","last_became_active":"2024-05-08T22:35:09.836290+0000","last_became_peered":"2024-05-08T22:35:09.836290+0000","last_unstale":"2024-05-08T22:35:11.840848+0000","last_undegraded":"2024-05-08T22:35:11.840848+0000","last_fullsized":"2024-05-08T22:35:11.840848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:13.029182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812403+0000","last_change":"2024-05-08T22:35:09.836148+0000","last_active":"2024-05-08T22:35:10.812403+0000","last_peered":"2024-05-08T22:35:10.812403+0000","last_clean":"2024-05-08T22:35:10.812403+0000","last_became_active":"2024-05-08T22:35:09.835767+0000","last_became_peered":"2024-05-08T22:35:09.835767+0000","last_unstale":"2024-05-08T22:35:10.812403+0000","last_undegraded":"2024-05-08T22:35:10.812403+0000","last_fullsized":"2024-05-08T22:35:10.812403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:23.343495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.841116+0000","last_change":"2024-05-08T22:35:09.840558+0000","last_active":"2024-05-08T22:35:09.841116+0000","last_peered":"2024-05-08T22:35:09.841116+0000","last_clean":"2024-05-08T22:35:09.841116+0000","last_became_active":"2024-05-08T22:35:09.840080+0000","last_became_peered":"2024-05-08T22:35:09.840080+0000","last_unstale":"2024-05-08T22:35:09.841116+0000","last_undegraded":"2024-05-08T22:35:09.841116+0000","last_fullsized":"2024-05-08T22:35:09.841116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:32.081217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.842689+0000","last_change":"2024-05-08T22:35:09.831151+0000","last_active":"2024-05-08T22:35:11.842689+0000","last_peered":"2024-05-08T22:35:11.842689+0000","last_clean":"2024-05-08T22:35:11.842689+0000","last_became_active":"2024-05-08T22:35:09.830693+0000","last_became_peered":"2024-05-08T22:35:09.830693+0000","last_unstale":"2024-05-08T22:35:11.842689+0000","last_undegraded":"2024-05-08T22:35:11.842689+0000","last_fullsized":"2024-05-08T22:35:11.842689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:15.060787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841080+0000","last_change":"2024-05-08T22:35:09.830620+0000","last_active":"2024-05-08T22:35:11.841080+0000","last_peered":"2024-05-08T22:35:11.841080+0000","last_clean":"2024-05-08T22:35:11.841080+0000","last_became_active":"2024-05-08T22:35:09.830289+0000","last_became_peered":"2024-05-08T22:35:09.830289+0000","last_unstale":"2024-05-08T22:35:11.841080+0000","last_undegraded":"2024-05-08T22:35:11.841080+0000","last_fullsized":"2024-05-08T22:35:11.841080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:49:15.181957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.847355+0000","last_change":"2024-05-08T22:35:09.391570+0000","last_active":"2024-05-08T22:35:13.847355+0000","last_peered":"2024-05-08T22:35:13.847355+0000","last_clean":"2024-05-08T22:35:13.847355+0000","last_became_active":"2024-05-08T22:35:09.391234+0000","last_became_peered":"2024-05-08T22:35:09.391234+0000","last_unstale":"2024-05-08T22:35:13.847355+0000","last_undegraded":"2024-05-08T22:35:13.847355+0000","last_fullsized":"2024-05-08T22:35:13.847355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:37.226688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825589+0000","last_change":"2024-05-08T22:35:09.831973+0000","last_active":"2024-05-08T22:35:11.825589+0000","last_peered":"2024-05-08T22:35:11.825589+0000","last_clean":"2024-05-08T22:35:11.825589+0000","last_became_active":"2024-05-08T22:35:09.829817+0000","last_became_peered":"2024-05-08T22:35:09.829817+0000","last_unstale":"2024-05-08T22:35:11.825589+0000","last_undegraded":"2024-05-08T22:35:11.825589+0000","last_fullsized":"2024-05-08T22:35:11.825589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:53.829103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841201+0000","last_change":"2024-05-08T22:35:09.393821+0000","last_active":"2024-05-08T22:35:11.841201+0000","last_peered":"2024-05-08T22:35:11.841201+0000","last_clean":"2024-05-08T22:35:11.841201+0000","last_became_active":"2024-05-08T22:35:09.393503+0000","last_became_peered":"2024-05-08T22:35:09.393503+0000","last_unstale":"2024-05-08T22:35:11.841201+0000","last_undegraded":"2024-05-08T22:35:11.841201+0000","last_fullsized":"2024-05-08T22:35:11.841201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:56.252015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841407+0000","last_change":"2024-05-08T22:35:09.823341+0000","last_active":"2024-05-08T22:35:11.841407+0000","last_peered":"2024-05-08T22:35:11.841407+0000","last_clean":"2024-05-08T22:35:11.841407+0000","last_became_active":"2024-05-08T22:35:09.822949+0000","last_became_peered":"2024-05-08T22:35:09.822949+0000","last_unstale":"2024-05-08T22:35:11.841407+0000","last_undegraded":"2024-05-08T22:35:11.841407+0000","last_fullsized":"2024-05-08T22:35:11.841407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:49.189563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.845906+0000","last_change":"2024-05-08T22:35:09.391129+0000","last_active":"2024-05-08T22:35:13.845906+0000","last_peered":"2024-05-08T22:35:13.845906+0000","last_clean":"2024-05-08T22:35:13.845906+0000","last_became_active":"2024-05-08T22:35:09.390429+0000","last_became_peered":"2024-05-08T22:35:09.390429+0000","last_unstale":"2024-05-08T22:35:13.845906+0000","last_undegraded":"2024-05-08T22:35:13.845906+0000","last_fullsized":"2024-05-08T22:35:13.845906+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:18.416456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812741+0000","last_change":"2024-05-08T22:35:09.838274+0000","last_active":"2024-05-08T22:35:10.812741+0000","last_peered":"2024-05-08T22:35:10.812741+0000","last_clean":"2024-05-08T22:35:10.812741+0000","last_became_active":"2024-05-08T22:35:09.837637+0000","last_became_peered":"2024-05-08T22:35:09.837637+0000","last_unstale":"2024-05-08T22:35:10.812741+0000","last_undegraded":"2024-05-08T22:35:10.812741+0000","last_fullsized":"2024-05-08T22:35:10.812741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:00.476914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.832187+0000","last_change":"2024-05-08T22:35:09.391987+0000","last_active":"2024-05-08T22:35:09.832187+0000","last_peered":"2024-05-08T22:35:09.832187+0000","last_clean":"2024-05-08T22:35:09.832187+0000","last_became_active":"2024-05-08T22:35:09.390587+0000","last_became_peered":"2024-05-08T22:35:09.390587+0000","last_unstale":"2024-05-08T22:35:09.832187+0000","last_undegraded":"2024-05-08T22:35:09.832187+0000","last_fullsized":"2024-05-08T22:35:09.832187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:14.343341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813185+0000","last_change":"2024-05-08T22:35:09.823177+0000","last_active":"2024-05-08T22:35:10.813185+0000","last_peered":"2024-05-08T22:35:10.813185+0000","last_clean":"2024-05-08T22:35:10.813185+0000","last_became_active":"2024-05-08T22:35:09.822830+0000","last_became_peered":"2024-05-08T22:35:09.822830+0000","last_unstale":"2024-05-08T22:35:10.813185+0000","last_undegraded":"2024-05-08T22:35:10.813185+0000","last_fullsized":"2024-05-08T22:35:10.813185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:44.648818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:16.264577+0000","last_change":"2024-05-08T22:35:09.392558+0000","last_active":"2024-05-08T22:35:16.264577+0000","last_peered":"2024-05-08T22:35:16.264577+0000","last_clean":"2024-05-08T22:35:16.264577+0000","last_became_active":"2024-05-08T22:35:09.392235+0000","last_became_peered":"2024-05-08T22:35:09.392235+0000","last_unstale":"2024-05-08T22:35:16.264577+0000","last_undegraded":"2024-05-08T22:35:16.264577+0000","last_fullsized":"2024-05-08T22:35:16.264577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:47:12.186400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.830402+0000","last_change":"2024-05-08T22:35:09.823455+0000","last_active":"2024-05-08T22:35:09.830402+0000","last_peered":"2024-05-08T22:35:09.830402+0000","last_clean":"2024-05-08T22:35:09.830402+0000","last_became_active":"2024-05-08T22:35:09.822818+0000","last_became_peered":"2024-05-08T22:35:09.822818+0000","last_unstale":"2024-05-08T22:35:09.830402+0000","last_undegraded":"2024-05-08T22:35:09.830402+0000","last_fullsized":"2024-05-08T22:35:09.830402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:38.142265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:14.857485+0000","last_change":"2024-05-08T22:35:09.394713+0000","last_active":"2024-05-08T22:35:14.857485+0000","last_peered":"2024-05-08T22:35:14.857485+0000","last_clean":"2024-05-08T22:35:14.857485+0000","last_became_active":"2024-05-08T22:35:09.394397+0000","last_became_peered":"2024-05-08T22:35:09.394397+0000","last_unstale":"2024-05-08T22:35:14.857485+0000","last_undegraded":"2024-05-08T22:35:14.857485+0000","last_fullsized":"2024-05-08T22:35:14.857485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:58:30.594207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813065+0000","last_change":"2024-05-08T22:35:09.827921+0000","last_active":"2024-05-08T22:35:10.813065+0000","last_peered":"2024-05-08T22:35:10.813065+0000","last_clean":"2024-05-08T22:35:10.813065+0000","last_became_active":"2024-05-08T22:35:09.827387+0000","last_became_peered":"2024-05-08T22:35:09.827387+0000","last_unstale":"2024-05-08T22:35:10.813065+0000","last_undegraded":"2024-05-08T22:35:10.813065+0000","last_fullsized":"2024-05-08T22:35:10.813065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:33.986935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.845741+0000","last_change":"2024-05-08T22:35:09.384387+0000","last_active":"2024-05-08T22:35:13.845741+0000","last_peered":"2024-05-08T22:35:13.845741+0000","last_clean":"2024-05-08T22:35:13.845741+0000","last_became_active":"2024-05-08T22:35:09.383970+0000","last_became_peered":"2024-05-08T22:35:09.383970+0000","last_unstale":"2024-05-08T22:35:13.845741+0000","last_undegraded":"2024-05-08T22:35:13.845741+0000","last_fullsized":"2024-05-08T22:35:13.845741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:55.430960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825444+0000","last_change":"2024-05-08T22:35:09.831477+0000","last_active":"2024-05-08T22:35:11.825444+0000","last_peered":"2024-05-08T22:35:11.825444+0000","last_clean":"2024-05-08T22:35:11.825444+0000","last_became_active":"2024-05-08T22:35:09.831168+0000","last_became_peered":"2024-05-08T22:35:09.831168+0000","last_unstale":"2024-05-08T22:35:11.825444+0000","last_undegraded":"2024-05-08T22:35:11.825444+0000","last_fullsized":"2024-05-08T22:35:11.825444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:56.765210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267651+0000","last_change":"2024-05-08T22:35:09.389063+0000","last_active":"2024-05-08T22:35:15.267651+0000","last_peered":"2024-05-08T22:35:15.267651+0000","last_clean":"2024-05-08T22:35:15.267651+0000","last_became_active":"2024-05-08T22:35:09.388683+0000","last_became_peered":"2024-05-08T22:35:09.388683+0000","last_unstale":"2024-05-08T22:35:15.267651+0000","last_undegraded":"2024-05-08T22:35:15.267651+0000","last_fullsized":"2024-05-08T22:35:15.267651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:03.804688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.842741+0000","last_change":"2024-05-08T22:35:09.829277+0000","last_active":"2024-05-08T22:35:11.842741+0000","last_peered":"2024-05-08T22:35:11.842741+0000","last_clean":"2024-05-08T22:35:11.842741+0000","last_became_active":"2024-05-08T22:35:09.828782+0000","last_became_peered":"2024-05-08T22:35:09.828782+0000","last_unstale":"2024-05-08T22:35:11.842741+0000","last_undegraded":"2024-05-08T22:35:11.842741+0000","last_fullsized":"2024-05-08T22:35:11.842741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:47.012528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:16.264245+0000","last_change":"2024-05-08T22:35:09.387464+0000","last_active":"2024-05-08T22:35:16.264245+0000","last_peered":"2024-05-08T22:35:16.264245+0000","last_clean":"2024-05-08T22:35:16.264245+0000","last_became_active":"2024-05-08T22:35:09.386994+0000","last_became_peered":"2024-05-08T22:35:09.386994+0000","last_unstale":"2024-05-08T22:35:16.264245+0000","last_undegraded":"2024-05-08T22:35:16.264245+0000","last_fullsized":"2024-05-08T22:35:16.264245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:27:04.151479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812918+0000","last_change":"2024-05-08T22:35:09.830886+0000","last_active":"2024-05-08T22:35:10.812918+0000","last_peered":"2024-05-08T22:35:10.812918+0000","last_clean":"2024-05-08T22:35:10.812918+0000","last_became_active":"2024-05-08T22:35:09.830466+0000","last_became_peered":"2024-05-08T22:35:09.830466+0000","last_unstale":"2024-05-08T22:35:10.812918+0000","last_undegraded":"2024-05-08T22:35:10.812918+0000","last_fullsized":"2024-05-08T22:35:10.812918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:12.337934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267954+0000","last_change":"2024-05-08T22:35:09.391510+0000","last_active":"2024-05-08T22:35:15.267954+0000","last_peered":"2024-05-08T22:35:15.267954+0000","last_clean":"2024-05-08T22:35:15.267954+0000","last_became_active":"2024-05-08T22:35:09.391145+0000","last_became_peered":"2024-05-08T22:35:09.391145+0000","last_unstale":"2024-05-08T22:35:15.267954+0000","last_undegraded":"2024-05-08T22:35:15.267954+0000","last_fullsized":"2024-05-08T22:35:15.267954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:39.694122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841304+0000","last_change":"2024-05-08T22:35:09.837056+0000","last_active":"2024-05-08T22:35:11.841304+0000","last_peered":"2024-05-08T22:35:11.841304+0000","last_clean":"2024-05-08T22:35:11.841304+0000","last_became_active":"2024-05-08T22:35:09.836724+0000","last_became_peered":"2024-05-08T22:35:09.836724+0000","last_unstale":"2024-05-08T22:35:11.841304+0000","last_undegraded":"2024-05-08T22:35:11.841304+0000","last_fullsized":"2024-05-08T22:35:11.841304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:48.901289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:14.855791+0000","last_change":"2024-05-08T22:35:09.395826+0000","last_active":"2024-05-08T22:35:14.855791+0000","last_peered":"2024-05-08T22:35:14.855791+0000","last_clean":"2024-05-08T22:35:14.855791+0000","last_became_active":"2024-05-08T22:35:09.395330+0000","last_became_peered":"2024-05-08T22:35:09.395330+0000","last_unstale":"2024-05-08T22:35:14.855791+0000","last_undegraded":"2024-05-08T22:35:14.855791+0000","last_fullsized":"2024-05-08T22:35:14.855791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:43.577894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812839+0000","last_change":"2024-05-08T22:35:09.831625+0000","last_active":"2024-05-08T22:35:10.812839+0000","last_peered":"2024-05-08T22:35:10.812839+0000","last_clean":"2024-05-08T22:35:10.812839+0000","last_became_active":"2024-05-08T22:35:09.831310+0000","last_became_peered":"2024-05-08T22:35:09.831310+0000","last_unstale":"2024-05-08T22:35:10.812839+0000","last_undegraded":"2024-05-08T22:35:10.812839+0000","last_fullsized":"2024-05-08T22:35:10.812839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:16.743871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'21","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:16.534895+0000","last_change":"2024-05-08T22:35:09.389805+0000","last_active":"2024-05-08T22:35:16.534895+0000","last_peered":"2024-05-08T22:35:16.534895+0000","last_clean":"2024-05-08T22:35:16.534895+0000","last_became_active":"2024-05-08T22:35:09.389443+0000","last_became_peered":"2024-05-08T22:35:09.389443+0000","last_unstale":"2024-05-08T22:35:16.534895+0000","last_undegraded":"2024-05-08T22:35:16.534895+0000","last_fullsized":"2024-05-08T22:35:16.534895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:40.479503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38147,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":23,"num_write":23,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.835300+0000","last_change":"2024-05-08T22:35:09.834871+0000","last_active":"2024-05-08T22:35:09.835300+0000","last_peered":"2024-05-08T22:35:09.835300+0000","last_clean":"2024-05-08T22:35:09.835300+0000","last_became_active":"2024-05-08T22:35:09.834589+0000","last_became_peered":"2024-05-08T22:35:09.834589+0000","last_unstale":"2024-05-08T22:35:09.835300+0000","last_undegraded":"2024-05-08T22:35:09.835300+0000","last_fullsized":"2024-05-08T22:35:09.835300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:12.476142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.844535+0000","last_change":"2024-05-08T22:35:09.388615+0000","last_active":"2024-05-08T22:35:13.844535+0000","last_peered":"2024-05-08T22:35:13.844535+0000","last_clean":"2024-05-08T22:35:13.844535+0000","last_became_active":"2024-05-08T22:35:09.388272+0000","last_became_peered":"2024-05-08T22:35:09.388272+0000","last_unstale":"2024-05-08T22:35:13.844535+0000","last_undegraded":"2024-05-08T22:35:13.844535+0000","last_fullsized":"2024-05-08T22:35:13.844535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:23.565450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470024+0000","last_change":"2024-05-08T22:35:09.831511+0000","last_active":"2024-05-08T22:35:10.470024+0000","last_peered":"2024-05-08T22:35:10.470024+0000","last_clean":"2024-05-08T22:35:10.470024+0000","last_became_active":"2024-05-08T22:35:09.831216+0000","last_became_peered":"2024-05-08T22:35:09.831216+0000","last_unstale":"2024-05-08T22:35:10.470024+0000","last_undegraded":"2024-05-08T22:35:10.470024+0000","last_fullsized":"2024-05-08T22:35:10.470024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:46.244544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.268190+0000","last_change":"2024-05-08T22:35:09.391835+0000","last_active":"2024-05-08T22:35:15.268190+0000","last_peered":"2024-05-08T22:35:15.268190+0000","last_clean":"2024-05-08T22:35:15.268190+0000","last_became_active":"2024-05-08T22:35:09.391465+0000","last_became_peered":"2024-05-08T22:35:09.391465+0000","last_unstale":"2024-05-08T22:35:15.268190+0000","last_undegraded":"2024-05-08T22:35:15.268190+0000","last_fullsized":"2024-05-08T22:35:15.268190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:30.819323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623032+0000","last_change":"2024-05-08T22:35:09.840351+0000","last_active":"2024-05-08T22:35:11.623032+0000","last_peered":"2024-05-08T22:35:11.623032+0000","last_clean":"2024-05-08T22:35:11.623032+0000","last_became_active":"2024-05-08T22:35:09.840000+0000","last_became_peered":"2024-05-08T22:35:09.840000+0000","last_unstale":"2024-05-08T22:35:11.623032+0000","last_undegraded":"2024-05-08T22:35:11.623032+0000","last_fullsized":"2024-05-08T22:35:11.623032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:51.428513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.846470+0000","last_change":"2024-05-08T22:35:09.389572+0000","last_active":"2024-05-08T22:35:13.846470+0000","last_peered":"2024-05-08T22:35:13.846470+0000","last_clean":"2024-05-08T22:35:13.846470+0000","last_became_active":"2024-05-08T22:35:09.389245+0000","last_became_peered":"2024-05-08T22:35:09.389245+0000","last_unstale":"2024-05-08T22:35:13.846470+0000","last_undegraded":"2024-05-08T22:35:13.846470+0000","last_fullsized":"2024-05-08T22:35:13.846470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:04.992010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.622950+0000","last_change":"2024-05-08T22:35:09.840132+0000","last_active":"2024-05-08T22:35:11.622950+0000","last_peered":"2024-05-08T22:35:11.622950+0000","last_clean":"2024-05-08T22:35:11.622950+0000","last_became_active":"2024-05-08T22:35:09.839684+0000","last_became_peered":"2024-05-08T22:35:09.839684+0000","last_unstale":"2024-05-08T22:35:11.622950+0000","last_undegraded":"2024-05-08T22:35:11.622950+0000","last_fullsized":"2024-05-08T22:35:11.622950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:32.325406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.847302+0000","last_change":"2024-05-08T22:35:09.392220+0000","last_active":"2024-05-08T22:35:13.847302+0000","last_peered":"2024-05-08T22:35:13.847302+0000","last_clean":"2024-05-08T22:35:13.847302+0000","last_became_active":"2024-05-08T22:35:09.391889+0000","last_became_peered":"2024-05-08T22:35:09.391889+0000","last_unstale":"2024-05-08T22:35:13.847302+0000","last_undegraded":"2024-05-08T22:35:13.847302+0000","last_fullsized":"2024-05-08T22:35:13.847302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:01.417894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813015+0000","last_change":"2024-05-08T22:35:09.823395+0000","last_active":"2024-05-08T22:35:10.813015+0000","last_peered":"2024-05-08T22:35:10.813015+0000","last_clean":"2024-05-08T22:35:10.813015+0000","last_became_active":"2024-05-08T22:35:09.822484+0000","last_became_peered":"2024-05-08T22:35:09.822484+0000","last_unstale":"2024-05-08T22:35:10.813015+0000","last_undegraded":"2024-05-08T22:35:10.813015+0000","last_fullsized":"2024-05-08T22:35:10.813015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:35.330789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.829259+0000","last_change":"2024-05-08T22:35:09.379904+0000","last_active":"2024-05-08T22:35:09.829259+0000","last_peered":"2024-05-08T22:35:09.829259+0000","last_clean":"2024-05-08T22:35:09.829259+0000","last_became_active":"2024-05-08T22:35:09.379533+0000","last_became_peered":"2024-05-08T22:35:09.379533+0000","last_unstale":"2024-05-08T22:35:09.829259+0000","last_undegraded":"2024-05-08T22:35:09.829259+0000","last_fullsized":"2024-05-08T22:35:09.829259+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:14.625069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.838589+0000","last_change":"2024-05-08T22:35:09.836472+0000","last_active":"2024-05-08T22:35:09.838589+0000","last_peered":"2024-05-08T22:35:09.838589+0000","last_clean":"2024-05-08T22:35:09.838589+0000","last_became_active":"2024-05-08T22:35:09.836110+0000","last_became_peered":"2024-05-08T22:35:09.836110+0000","last_unstale":"2024-05-08T22:35:09.838589+0000","last_undegraded":"2024-05-08T22:35:09.838589+0000","last_fullsized":"2024-05-08T22:35:09.838589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:48.979661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.811775+0000","last_change":"2024-05-08T22:35:09.391647+0000","last_active":"2024-05-08T22:35:09.811775+0000","last_peered":"2024-05-08T22:35:09.811775+0000","last_clean":"2024-05-08T22:35:09.811775+0000","last_became_active":"2024-05-08T22:35:09.391273+0000","last_became_peered":"2024-05-08T22:35:09.391273+0000","last_unstale":"2024-05-08T22:35:09.811775+0000","last_undegraded":"2024-05-08T22:35:09.811775+0000","last_fullsized":"2024-05-08T22:35:09.811775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:48.001313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.752901+0000","last_change":"2024-05-08T22:35:09.828713+0000","last_active":"2024-05-08T22:35:11.752901+0000","last_peered":"2024-05-08T22:35:11.752901+0000","last_clean":"2024-05-08T22:35:11.752901+0000","last_became_active":"2024-05-08T22:35:09.828370+0000","last_became_peered":"2024-05-08T22:35:09.828370+0000","last_unstale":"2024-05-08T22:35:11.752901+0000","last_undegraded":"2024-05-08T22:35:11.752901+0000","last_fullsized":"2024-05-08T22:35:11.752901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:23.328029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:16.285247+0000","last_change":"2024-05-08T22:35:09.391658+0000","last_active":"2024-05-08T22:35:16.285247+0000","last_peered":"2024-05-08T22:35:16.285247+0000","last_clean":"2024-05-08T22:35:16.285247+0000","last_became_active":"2024-05-08T22:35:09.391326+0000","last_became_peered":"2024-05-08T22:35:09.391326+0000","last_unstale":"2024-05-08T22:35:16.285247+0000","last_undegraded":"2024-05-08T22:35:16.285247+0000","last_fullsized":"2024-05-08T22:35:16.285247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:08:13.933145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604425+0000","last_change":"2024-05-08T22:35:09.837816+0000","last_active":"2024-05-08T22:35:11.604425+0000","last_peered":"2024-05-08T22:35:11.604425+0000","last_clean":"2024-05-08T22:35:11.604425+0000","last_became_active":"2024-05-08T22:35:09.837471+0000","last_became_peered":"2024-05-08T22:35:09.837471+0000","last_unstale":"2024-05-08T22:35:11.604425+0000","last_undegraded":"2024-05-08T22:35:11.604425+0000","last_fullsized":"2024-05-08T22:35:11.604425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:04.564055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813006+0000","last_change":"2024-05-08T22:35:09.827092+0000","last_active":"2024-05-08T22:35:10.813006+0000","last_peered":"2024-05-08T22:35:10.813006+0000","last_clean":"2024-05-08T22:35:10.813006+0000","last_became_active":"2024-05-08T22:35:09.826238+0000","last_became_peered":"2024-05-08T22:35:09.826238+0000","last_unstale":"2024-05-08T22:35:10.813006+0000","last_undegraded":"2024-05-08T22:35:10.813006+0000","last_fullsized":"2024-05-08T22:35:10.813006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:26:59.504555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.259760+0000","last_change":"2024-05-08T22:35:09.391555+0000","last_active":"2024-05-08T22:35:15.259760+0000","last_peered":"2024-05-08T22:35:15.259760+0000","last_clean":"2024-05-08T22:35:15.259760+0000","last_became_active":"2024-05-08T22:35:09.391294+0000","last_became_peered":"2024-05-08T22:35:09.391294+0000","last_unstale":"2024-05-08T22:35:15.259760+0000","last_undegraded":"2024-05-08T22:35:15.259760+0000","last_fullsized":"2024-05-08T22:35:15.259760+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:41.195882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825544+0000","last_change":"2024-05-08T22:35:09.822814+0000","last_active":"2024-05-08T22:35:11.825544+0000","last_peered":"2024-05-08T22:35:11.825544+0000","last_clean":"2024-05-08T22:35:11.825544+0000","last_became_active":"2024-05-08T22:35:09.822425+0000","last_became_peered":"2024-05-08T22:35:09.822425+0000","last_unstale":"2024-05-08T22:35:11.825544+0000","last_undegraded":"2024-05-08T22:35:11.825544+0000","last_fullsized":"2024-05-08T22:35:11.825544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:56.212615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.839323+0000","last_change":"2024-05-08T22:35:09.383782+0000","last_active":"2024-05-08T22:35:12.839323+0000","last_peered":"2024-05-08T22:35:12.839323+0000","last_clean":"2024-05-08T22:35:12.839323+0000","last_became_active":"2024-05-08T22:35:09.383285+0000","last_became_peered":"2024-05-08T22:35:09.383285+0000","last_unstale":"2024-05-08T22:35:12.839323+0000","last_undegraded":"2024-05-08T22:35:12.839323+0000","last_fullsized":"2024-05-08T22:35:12.839323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:42.380683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":322,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.822963+0000","last_change":"2024-05-08T22:34:20.037406+0000","last_active":"2024-05-08T22:35:09.822963+0000","last_peered":"2024-05-08T22:35:09.822963+0000","last_clean":"2024-05-08T22:35:09.822963+0000","last_became_active":"2024-05-08T22:34:20.036880+0000","last_became_peered":"2024-05-08T22:34:20.036880+0000","last_unstale":"2024-05-08T22:35:09.822963+0000","last_undegraded":"2024-05-08T22:35:09.822963+0000","last_fullsized":"2024-05-08T22:35:09.822963+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:31:54.953551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:31:54.953551+0000","last_clean_scrub_stamp":"2024-05-08T22:31:54.953551+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:00.820169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.469956+0000","last_change":"2024-05-08T22:35:09.830425+0000","last_active":"2024-05-08T22:35:10.469956+0000","last_peered":"2024-05-08T22:35:10.469956+0000","last_clean":"2024-05-08T22:35:10.469956+0000","last_became_active":"2024-05-08T22:35:09.830097+0000","last_became_peered":"2024-05-08T22:35:09.830097+0000","last_unstale":"2024-05-08T22:35:10.469956+0000","last_undegraded":"2024-05-08T22:35:10.469956+0000","last_fullsized":"2024-05-08T22:35:10.469956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:53.445798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.846273+0000","last_change":"2024-05-08T22:35:09.386717+0000","last_active":"2024-05-08T22:35:13.846273+0000","last_peered":"2024-05-08T22:35:13.846273+0000","last_clean":"2024-05-08T22:35:13.846273+0000","last_became_active":"2024-05-08T22:35:09.386371+0000","last_became_peered":"2024-05-08T22:35:09.386371+0000","last_unstale":"2024-05-08T22:35:13.846273+0000","last_undegraded":"2024-05-08T22:35:13.846273+0000","last_fullsized":"2024-05-08T22:35:13.846273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:41.061997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.816655+0000","last_change":"2024-05-08T22:35:09.394542+0000","last_active":"2024-05-08T22:35:10.816655+0000","last_peered":"2024-05-08T22:35:10.816655+0000","last_clean":"2024-05-08T22:35:10.816655+0000","last_became_active":"2024-05-08T22:35:09.394223+0000","last_became_peered":"2024-05-08T22:35:09.394223+0000","last_unstale":"2024-05-08T22:35:10.816655+0000","last_undegraded":"2024-05-08T22:35:10.816655+0000","last_fullsized":"2024-05-08T22:35:10.816655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:02.162377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470081+0000","last_change":"2024-05-08T22:35:09.825810+0000","last_active":"2024-05-08T22:35:10.470081+0000","last_peered":"2024-05-08T22:35:10.470081+0000","last_clean":"2024-05-08T22:35:10.470081+0000","last_became_active":"2024-05-08T22:35:09.825516+0000","last_became_peered":"2024-05-08T22:35:09.825516+0000","last_unstale":"2024-05-08T22:35:10.470081+0000","last_undegraded":"2024-05-08T22:35:10.470081+0000","last_fullsized":"2024-05-08T22:35:10.470081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:24.087187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.847375+0000","last_change":"2024-05-08T22:35:09.381688+0000","last_active":"2024-05-08T22:35:13.847375+0000","last_peered":"2024-05-08T22:35:13.847375+0000","last_clean":"2024-05-08T22:35:13.847375+0000","last_became_active":"2024-05-08T22:35:09.381343+0000","last_became_peered":"2024-05-08T22:35:09.381343+0000","last_unstale":"2024-05-08T22:35:13.847375+0000","last_undegraded":"2024-05-08T22:35:13.847375+0000","last_fullsized":"2024-05-08T22:35:13.847375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:14:16.142265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825676+0000","last_change":"2024-05-08T22:35:09.831689+0000","last_active":"2024-05-08T22:35:11.825676+0000","last_peered":"2024-05-08T22:35:11.825676+0000","last_clean":"2024-05-08T22:35:11.825676+0000","last_became_active":"2024-05-08T22:35:09.831337+0000","last_became_peered":"2024-05-08T22:35:09.831337+0000","last_unstale":"2024-05-08T22:35:11.825676+0000","last_undegraded":"2024-05-08T22:35:11.825676+0000","last_fullsized":"2024-05-08T22:35:11.825676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.965340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.839504+0000","last_change":"2024-05-08T22:35:09.390479+0000","last_active":"2024-05-08T22:35:12.839504+0000","last_peered":"2024-05-08T22:35:12.839504+0000","last_clean":"2024-05-08T22:35:12.839504+0000","last_became_active":"2024-05-08T22:35:09.389790+0000","last_became_peered":"2024-05-08T22:35:09.389790+0000","last_unstale":"2024-05-08T22:35:12.839504+0000","last_undegraded":"2024-05-08T22:35:12.839504+0000","last_fullsized":"2024-05-08T22:35:12.839504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:27.668801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841685+0000","last_change":"2024-05-08T22:35:09.836123+0000","last_active":"2024-05-08T22:35:11.841685+0000","last_peered":"2024-05-08T22:35:11.841685+0000","last_clean":"2024-05-08T22:35:11.841685+0000","last_became_active":"2024-05-08T22:35:09.835840+0000","last_became_peered":"2024-05-08T22:35:09.835840+0000","last_unstale":"2024-05-08T22:35:11.841685+0000","last_undegraded":"2024-05-08T22:35:11.841685+0000","last_fullsized":"2024-05-08T22:35:11.841685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:52.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.840918+0000","last_change":"2024-05-08T22:35:09.394658+0000","last_active":"2024-05-08T22:35:12.840918+0000","last_peered":"2024-05-08T22:35:12.840918+0000","last_clean":"2024-05-08T22:35:12.840918+0000","last_became_active":"2024-05-08T22:35:09.394351+0000","last_became_peered":"2024-05-08T22:35:09.394351+0000","last_unstale":"2024-05-08T22:35:12.840918+0000","last_undegraded":"2024-05-08T22:35:12.840918+0000","last_fullsized":"2024-05-08T22:35:12.840918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:32.954612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.834139+0000","last_change":"2024-05-08T22:35:09.830853+0000","last_active":"2024-05-08T22:35:09.834139+0000","last_peered":"2024-05-08T22:35:09.834139+0000","last_clean":"2024-05-08T22:35:09.834139+0000","last_became_active":"2024-05-08T22:35:09.830471+0000","last_became_peered":"2024-05-08T22:35:09.830471+0000","last_unstale":"2024-05-08T22:35:09.834139+0000","last_undegraded":"2024-05-08T22:35:09.834139+0000","last_fullsized":"2024-05-08T22:35:09.834139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:13.960966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.265416+0000","last_change":"2024-05-08T22:35:09.382515+0000","last_active":"2024-05-08T22:35:15.265416+0000","last_peered":"2024-05-08T22:35:15.265416+0000","last_clean":"2024-05-08T22:35:15.265416+0000","last_became_active":"2024-05-08T22:35:09.382168+0000","last_became_peered":"2024-05-08T22:35:09.382168+0000","last_unstale":"2024-05-08T22:35:15.265416+0000","last_undegraded":"2024-05-08T22:35:15.265416+0000","last_fullsized":"2024-05-08T22:35:15.265416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:10.248542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.837958+0000","last_change":"2024-05-08T22:35:09.835925+0000","last_active":"2024-05-08T22:35:09.837958+0000","last_peered":"2024-05-08T22:35:09.837958+0000","last_clean":"2024-05-08T22:35:09.837958+0000","last_became_active":"2024-05-08T22:35:09.835547+0000","last_became_peered":"2024-05-08T22:35:09.835547+0000","last_unstale":"2024-05-08T22:35:09.837958+0000","last_undegraded":"2024-05-08T22:35:09.837958+0000","last_fullsized":"2024-05-08T22:35:09.837958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:36.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841302+0000","last_change":"2024-05-08T22:35:09.385035+0000","last_active":"2024-05-08T22:35:11.841302+0000","last_peered":"2024-05-08T22:35:11.841302+0000","last_clean":"2024-05-08T22:35:11.841302+0000","last_became_active":"2024-05-08T22:35:09.384274+0000","last_became_peered":"2024-05-08T22:35:09.384274+0000","last_unstale":"2024-05-08T22:35:11.841302+0000","last_undegraded":"2024-05-08T22:35:11.841302+0000","last_fullsized":"2024-05-08T22:35:11.841302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:50.654985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825567+0000","last_change":"2024-05-08T22:35:09.821454+0000","last_active":"2024-05-08T22:35:11.825567+0000","last_peered":"2024-05-08T22:35:11.825567+0000","last_clean":"2024-05-08T22:35:11.825567+0000","last_became_active":"2024-05-08T22:35:09.821199+0000","last_became_peered":"2024-05-08T22:35:09.821199+0000","last_unstale":"2024-05-08T22:35:11.825567+0000","last_undegraded":"2024-05-08T22:35:11.825567+0000","last_fullsized":"2024-05-08T22:35:11.825567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:02.677307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.840347+0000","last_change":"2024-05-08T22:35:09.394027+0000","last_active":"2024-05-08T22:35:12.840347+0000","last_peered":"2024-05-08T22:35:12.840347+0000","last_clean":"2024-05-08T22:35:12.840347+0000","last_became_active":"2024-05-08T22:35:09.393622+0000","last_became_peered":"2024-05-08T22:35:09.393622+0000","last_unstale":"2024-05-08T22:35:12.840347+0000","last_undegraded":"2024-05-08T22:35:12.840347+0000","last_fullsized":"2024-05-08T22:35:12.840347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:13.415635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623012+0000","last_change":"2024-05-08T22:35:09.842928+0000","last_active":"2024-05-08T22:35:11.623012+0000","last_peered":"2024-05-08T22:35:11.623012+0000","last_clean":"2024-05-08T22:35:11.623012+0000","last_became_active":"2024-05-08T22:35:09.842576+0000","last_became_peered":"2024-05-08T22:35:09.842576+0000","last_unstale":"2024-05-08T22:35:11.623012+0000","last_undegraded":"2024-05-08T22:35:11.623012+0000","last_fullsized":"2024-05-08T22:35:11.623012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:03.526259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.846645+0000","last_change":"2024-05-08T22:35:09.396325+0000","last_active":"2024-05-08T22:35:13.846645+0000","last_peered":"2024-05-08T22:35:13.846645+0000","last_clean":"2024-05-08T22:35:13.846645+0000","last_became_active":"2024-05-08T22:35:09.395656+0000","last_became_peered":"2024-05-08T22:35:09.395656+0000","last_unstale":"2024-05-08T22:35:13.846645+0000","last_undegraded":"2024-05-08T22:35:13.846645+0000","last_fullsized":"2024-05-08T22:35:13.846645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:30.326694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623076+0000","last_change":"2024-05-08T22:35:09.840682+0000","last_active":"2024-05-08T22:35:11.623076+0000","last_peered":"2024-05-08T22:35:11.623076+0000","last_clean":"2024-05-08T22:35:11.623076+0000","last_became_active":"2024-05-08T22:35:09.840315+0000","last_became_peered":"2024-05-08T22:35:09.840315+0000","last_unstale":"2024-05-08T22:35:11.623076+0000","last_undegraded":"2024-05-08T22:35:11.623076+0000","last_fullsized":"2024-05-08T22:35:11.623076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:13.884569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.257932+0000","last_change":"2024-05-08T22:35:09.389169+0000","last_active":"2024-05-08T22:35:15.257932+0000","last_peered":"2024-05-08T22:35:15.257932+0000","last_clean":"2024-05-08T22:35:15.257932+0000","last_became_active":"2024-05-08T22:35:09.388836+0000","last_became_peered":"2024-05-08T22:35:09.388836+0000","last_unstale":"2024-05-08T22:35:15.257932+0000","last_undegraded":"2024-05-08T22:35:15.257932+0000","last_fullsized":"2024-05-08T22:35:15.257932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:08.960719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841499+0000","last_change":"2024-05-08T22:35:09.833885+0000","last_active":"2024-05-08T22:35:11.841499+0000","last_peered":"2024-05-08T22:35:11.841499+0000","last_clean":"2024-05-08T22:35:11.841499+0000","last_became_active":"2024-05-08T22:35:09.833596+0000","last_became_peered":"2024-05-08T22:35:09.833596+0000","last_unstale":"2024-05-08T22:35:11.841499+0000","last_undegraded":"2024-05-08T22:35:11.841499+0000","last_fullsized":"2024-05-08T22:35:11.841499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:29.673837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267349+0000","last_change":"2024-05-08T22:35:09.389023+0000","last_active":"2024-05-08T22:35:15.267349+0000","last_peered":"2024-05-08T22:35:15.267349+0000","last_clean":"2024-05-08T22:35:15.267349+0000","last_became_active":"2024-05-08T22:35:09.388721+0000","last_became_peered":"2024-05-08T22:35:09.388721+0000","last_unstale":"2024-05-08T22:35:15.267349+0000","last_undegraded":"2024-05-08T22:35:15.267349+0000","last_fullsized":"2024-05-08T22:35:15.267349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:41.776592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623109+0000","last_change":"2024-05-08T22:35:09.839499+0000","last_active":"2024-05-08T22:35:11.623109+0000","last_peered":"2024-05-08T22:35:11.623109+0000","last_clean":"2024-05-08T22:35:11.623109+0000","last_became_active":"2024-05-08T22:35:09.839146+0000","last_became_peered":"2024-05-08T22:35:09.839146+0000","last_unstale":"2024-05-08T22:35:11.623109+0000","last_undegraded":"2024-05-08T22:35:11.623109+0000","last_fullsized":"2024-05-08T22:35:11.623109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:45.636827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.839015+0000","last_change":"2024-05-08T22:35:09.836710+0000","last_active":"2024-05-08T22:35:09.839015+0000","last_peered":"2024-05-08T22:35:09.839015+0000","last_clean":"2024-05-08T22:35:09.839015+0000","last_became_active":"2024-05-08T22:35:09.835838+0000","last_became_peered":"2024-05-08T22:35:09.835838+0000","last_unstale":"2024-05-08T22:35:09.839015+0000","last_undegraded":"2024-05-08T22:35:09.839015+0000","last_fullsized":"2024-05-08T22:35:09.839015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:00.567805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841269+0000","last_change":"2024-05-08T22:35:09.830148+0000","last_active":"2024-05-08T22:35:11.841269+0000","last_peered":"2024-05-08T22:35:11.841269+0000","last_clean":"2024-05-08T22:35:11.841269+0000","last_became_active":"2024-05-08T22:35:09.829820+0000","last_became_peered":"2024-05-08T22:35:09.829820+0000","last_unstale":"2024-05-08T22:35:11.841269+0000","last_undegraded":"2024-05-08T22:35:11.841269+0000","last_fullsized":"2024-05-08T22:35:11.841269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:38.840993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841227+0000","last_change":"2024-05-08T22:35:09.830428+0000","last_active":"2024-05-08T22:35:11.841227+0000","last_peered":"2024-05-08T22:35:11.841227+0000","last_clean":"2024-05-08T22:35:11.841227+0000","last_became_active":"2024-05-08T22:35:09.829736+0000","last_became_peered":"2024-05-08T22:35:09.829736+0000","last_unstale":"2024-05-08T22:35:11.841227+0000","last_undegraded":"2024-05-08T22:35:11.841227+0000","last_fullsized":"2024-05-08T22:35:11.841227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:54.458662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841649+0000","last_change":"2024-05-08T22:35:09.836200+0000","last_active":"2024-05-08T22:35:11.841649+0000","last_peered":"2024-05-08T22:35:11.841649+0000","last_clean":"2024-05-08T22:35:11.841649+0000","last_became_active":"2024-05-08T22:35:09.835892+0000","last_became_peered":"2024-05-08T22:35:09.835892+0000","last_unstale":"2024-05-08T22:35:11.841649+0000","last_undegraded":"2024-05-08T22:35:11.841649+0000","last_fullsized":"2024-05-08T22:35:11.841649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:42.487982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470102+0000","last_change":"2024-05-08T22:35:09.834026+0000","last_active":"2024-05-08T22:35:10.470102+0000","last_peered":"2024-05-08T22:35:10.470102+0000","last_clean":"2024-05-08T22:35:10.470102+0000","last_became_active":"2024-05-08T22:35:09.833659+0000","last_became_peered":"2024-05-08T22:35:09.833659+0000","last_unstale":"2024-05-08T22:35:10.470102+0000","last_undegraded":"2024-05-08T22:35:10.470102+0000","last_fullsized":"2024-05-08T22:35:10.470102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:00.536853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.839061+0000","last_change":"2024-05-08T22:35:09.836928+0000","last_active":"2024-05-08T22:35:09.839061+0000","last_peered":"2024-05-08T22:35:09.839061+0000","last_clean":"2024-05-08T22:35:09.839061+0000","last_became_active":"2024-05-08T22:35:09.836629+0000","last_became_peered":"2024-05-08T22:35:09.836629+0000","last_unstale":"2024-05-08T22:35:09.839061+0000","last_undegraded":"2024-05-08T22:35:09.839061+0000","last_fullsized":"2024-05-08T22:35:09.839061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:43.475121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841544+0000","last_change":"2024-05-08T22:35:09.837685+0000","last_active":"2024-05-08T22:35:11.841544+0000","last_peered":"2024-05-08T22:35:11.841544+0000","last_clean":"2024-05-08T22:35:11.841544+0000","last_became_active":"2024-05-08T22:35:09.837409+0000","last_became_peered":"2024-05-08T22:35:09.837409+0000","last_unstale":"2024-05-08T22:35:11.841544+0000","last_undegraded":"2024-05-08T22:35:11.841544+0000","last_fullsized":"2024-05-08T22:35:11.841544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:32.076361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825653+0000","last_change":"2024-05-08T22:35:09.833099+0000","last_active":"2024-05-08T22:35:11.825653+0000","last_peered":"2024-05-08T22:35:11.825653+0000","last_clean":"2024-05-08T22:35:11.825653+0000","last_became_active":"2024-05-08T22:35:09.832814+0000","last_became_peered":"2024-05-08T22:35:09.832814+0000","last_unstale":"2024-05-08T22:35:11.825653+0000","last_undegraded":"2024-05-08T22:35:11.825653+0000","last_fullsized":"2024-05-08T22:35:11.825653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:40.116060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812821+0000","last_change":"2024-05-08T22:35:09.818507+0000","last_active":"2024-05-08T22:35:10.812821+0000","last_peered":"2024-05-08T22:35:10.812821+0000","last_clean":"2024-05-08T22:35:10.812821+0000","last_became_active":"2024-05-08T22:35:09.818139+0000","last_became_peered":"2024-05-08T22:35:09.818139+0000","last_unstale":"2024-05-08T22:35:10.812821+0000","last_undegraded":"2024-05-08T22:35:10.812821+0000","last_fullsized":"2024-05-08T22:35:10.812821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:33.747011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825503+0000","last_change":"2024-05-08T22:35:09.832232+0000","last_active":"2024-05-08T22:35:11.825503+0000","last_peered":"2024-05-08T22:35:11.825503+0000","last_clean":"2024-05-08T22:35:11.825503+0000","last_became_active":"2024-05-08T22:35:09.831889+0000","last_became_peered":"2024-05-08T22:35:09.831889+0000","last_unstale":"2024-05-08T22:35:11.825503+0000","last_undegraded":"2024-05-08T22:35:11.825503+0000","last_fullsized":"2024-05-08T22:35:11.825503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:01.802009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812645+0000","last_change":"2024-05-08T22:35:09.838051+0000","last_active":"2024-05-08T22:35:10.812645+0000","last_peered":"2024-05-08T22:35:10.812645+0000","last_clean":"2024-05-08T22:35:10.812645+0000","last_became_active":"2024-05-08T22:35:09.837429+0000","last_became_peered":"2024-05-08T22:35:09.837429+0000","last_unstale":"2024-05-08T22:35:10.812645+0000","last_undegraded":"2024-05-08T22:35:10.812645+0000","last_fullsized":"2024-05-08T22:35:10.812645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.191374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470060+0000","last_change":"2024-05-08T22:35:09.830834+0000","last_active":"2024-05-08T22:35:10.470060+0000","last_peered":"2024-05-08T22:35:10.470060+0000","last_clean":"2024-05-08T22:35:10.470060+0000","last_became_active":"2024-05-08T22:35:09.830475+0000","last_became_peered":"2024-05-08T22:35:09.830475+0000","last_unstale":"2024-05-08T22:35:10.470060+0000","last_undegraded":"2024-05-08T22:35:10.470060+0000","last_fullsized":"2024-05-08T22:35:10.470060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:06:10.996668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.838544+0000","last_change":"2024-05-08T22:35:09.836496+0000","last_active":"2024-05-08T22:35:09.838544+0000","last_peered":"2024-05-08T22:35:09.838544+0000","last_clean":"2024-05-08T22:35:09.838544+0000","last_became_active":"2024-05-08T22:35:09.835787+0000","last_became_peered":"2024-05-08T22:35:09.835787+0000","last_unstale":"2024-05-08T22:35:09.838544+0000","last_undegraded":"2024-05-08T22:35:09.838544+0000","last_fullsized":"2024-05-08T22:35:09.838544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:30.395286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43378,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":24,"num_read_kb":37,"num_write":169,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":94426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":856064,"data_stored":410021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]}]},{"osd":10,"up_from":67,"seq":287762808847,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32596,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712556,"statfs":{"total":95995035648,"available":95961657344,"internally_reserved":0,"allocated":1519616,"data_stored":1022487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]}]},{"osd":9,"up_from":62,"seq":266287972370,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":872448,"data_stored":410257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.45700000000000002}]},{"osd":4,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53800000000000003}]},{"osd":5,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.55700000000000005}]},{"osd":6,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.496}]},{"osd":7,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.70499999999999996}]},{"osd":8,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.46700000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":8,"up_from":55,"seq":236223201301,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":868352,"data_stored":409896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.438}]},{"osd":1,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61399999999999999}]},{"osd":3,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":5.468,"5min":5.468,"15min":5.468},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.79800000000000004}]},{"osd":5,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.71099999999999997}]},{"osd":6,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":7.3689999999999998,"5min":7.3689999999999998,"15min":7.3689999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.5620000000000003,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.68999999999999995}]},{"osd":7,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.88600000000000001}]},{"osd":9,"last update":"Wed May 8 22:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.751}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":7,"up_from":48,"seq":206158430233,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32552,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712600,"statfs":{"total":95995035648,"available":95961702400,"internally_reserved":0,"allocated":1474560,"data_stored":1003260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40100000000000002}]},{"osd":1,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.78000000000000003}]},{"osd":3,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.65000000000000002}]},{"osd":4,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.755}]},{"osd":5,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.82999999999999996}]},{"osd":6,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.84699999999999998}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.89800000000000002}]},{"osd":9,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.93899999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":6,"up_from":43,"seq":184683593756,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":409551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39000000000000001,"15min":0.38800000000000001},"min":{"1min":0.29399999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42499999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.76500000000000001}]},{"osd":1,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.82399999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77000000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.754},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.86299999999999999}]},{"osd":3,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47699999999999998,"15min":0.48599999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56999999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.49299999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.72899999999999998}]},{"osd":4,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63},"min":{"1min":0.44800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77500000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.874,"15min":0.874},"last":0.66500000000000004}]},{"osd":5,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.48199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":1.0369999999999999}]},{"osd":7,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.71299999999999997}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":1.0149999999999999}]},{"osd":9,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.95499999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36076,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709076,"statfs":{"total":95995035648,"available":95958093824,"internally_reserved":0,"allocated":888832,"data_stored":431719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48399999999999999,"15min":0.47199999999999998},"min":{"1min":0.317,"5min":0.311,"15min":0.311},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46300000000000002}]},{"osd":2,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83599999999999997,"5min":1.627,"15min":1.627},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.753}]},{"osd":3,"last update":"Wed May 8 22:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.35299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.621},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.40600000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.89300000000000002,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.53700000000000003}]},{"osd":4,"last update":"Wed May 8 22:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42499999999999999,"15min":0.41799999999999998},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.45100000000000001,"15min":0.435},"min":{"1min":0.32600000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.55700000000000005}]},{"osd":5,"last update":"Wed May 8 22:34:52 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.83399999999999996}]},{"osd":6,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.47599999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66600000000000004}]},{"osd":8,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.80300000000000005}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65100000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738418,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":864256,"data_stored":410133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.40600000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.52100000000000002,"15min":0.499},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.58599999999999997}]},{"osd":2,"last update":"Wed May 8 22:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.40200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51400000000000001,"15min":0.48799999999999999},"min":{"1min":0.36099999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.76500000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.56599999999999995}]},{"osd":3,"last update":"Wed May 8 22:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.39600000000000002,"15min":0.38700000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.42699999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45300000000000001}]},{"osd":5,"last update":"Wed May 8 22:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70599999999999996,"5min":0.751,"15min":0.751},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90000000000000002,"5min":1.085,"15min":1.085},"last":0.52600000000000002}]},{"osd":6,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78700000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83899999999999997,"5min":0.872,"15min":0.872},"last":0.377}]},{"osd":7,"last update":"Wed May 8 22:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53500000000000003}]},{"osd":8,"last update":"Wed May 8 22:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.433}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.443},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.48199999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411370,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":872448,"data_stored":410942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.502},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71799999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.72499999999999998}]},{"osd":1,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.40500000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.35799999999999998}]},{"osd":3,"last update":"Wed May 8 22:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.496},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73399999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed May 8 22:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.44},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.33200000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64100000000000001,"5min":0.748,"15min":0.748},"last":0.33400000000000002}]},{"osd":6,"last update":"Wed May 8 22:35:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.50600000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed May 8 22:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.47299999999999998}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.74399999999999999}]},{"osd":9,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.627},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.66900000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]}]},{"osd":3,"up_from":24,"seq":103079215143,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36072,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709080,"statfs":{"total":95995035648,"available":95958097920,"internally_reserved":0,"allocated":884736,"data_stored":409986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.33600000000000002},"min":{"1min":0.27200000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.54100000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.318},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.36399999999999999,"15min":0.35199999999999998},"min":{"1min":0.28899999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58899999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.51400000000000001}]},{"osd":1,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72999999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54100000000000004,"15min":0.52800000000000002},"min":{"1min":0.38800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.41799999999999998}]},{"osd":2,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88100000000000001,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.50700000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.47599999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53200000000000003}]},{"osd":5,"last update":"Wed May 8 22:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85499999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.48499999999999999}]},{"osd":6,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.435}]},{"osd":7,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56399999999999995}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.496},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.53900000000000003}]},{"osd":9,"last update":"Wed May 8 22:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.67700000000000005}]},{"osd":10,"last update":"Wed May 8 22:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.501},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.58899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":864256,"data_stored":410035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.38100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.42699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.39900000000000002},"min":{"1min":0.27000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77700000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.26400000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.63800000000000001,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75700000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.77200000000000002}]},{"osd":3,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.35499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89200000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.79700000000000004}]},{"osd":6,"last update":"Wed May 8 22:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.40699999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.747},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.82499999999999996}]},{"osd":7,"last update":"Wed May 8 22:34:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59399999999999997}]},{"osd":8,"last update":"Wed May 8 22:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.997}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.95199999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1490944,"data_stored":1002758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.50800000000000001,"15min":0.499},"min":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.623},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.35899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51800000000000002}]},{"osd":1,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.65400000000000003}]},{"osd":2,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.379,"15min":0.372},"min":{"1min":0.28199999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.374}]},{"osd":3,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed May 8 22:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40899999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68999999999999995}]},{"osd":7,"last update":"Wed May 8 22:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76200000000000001}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39400000000000002}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.72699999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":39851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":40228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:35:18.517 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T22:35:18.517 INFO:tasks.cephadm:Setting up client nodes... 2024-05-08T22:35:18.518 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-08T22:35:18.775 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.811+0000 7f0f01c2f700 1 -- 172.21.15.62:0/3525965991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101a50 msgr2=0x7f0efc103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.811+0000 7f0f01c2f700 1 --2- 172.21.15.62:0/3525965991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101a50 0x7f0efc103e40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009990 tx=0x7f0eec009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.811+0000 7f0f01c2f700 1 -- 172.21.15.62:0/3525965991 shutdown_connections 2024-05-08T22:35:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.811+0000 7f0f01c2f700 1 --2- 172.21.15.62:0/3525965991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0efc106cb0 0x7f0efc109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.811+0000 7f0f01c2f700 1 --2- 172.21.15.62:0/3525965991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc101a50 0x7f0efc103e40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.811+0000 7f0f01c2f700 1 --2- 172.21.15.62:0/3525965991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0efc104380 0x7f0efc106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.811+0000 7f0f01c2f700 1 -- 172.21.15.62:0/3525965991 >> 172.21.15.62:0/3525965991 conn(0x7f0efc0fb460 msgr2=0x7f0efc0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.811+0000 7f0f01c2f700 1 -- 172.21.15.62:0/3525965991 shutdown_connections 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.811+0000 7f0f01c2f700 1 -- 172.21.15.62:0/3525965991 wait complete. 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0f01c2f700 1 Processor -- start 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0f01c2f700 1 -- start start 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0f01c2f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0efc101a50 0x7f0efc196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0f01c2f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc104380 0x7f0efc197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0f01c2f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0efc106cb0 0x7f0efc19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0f01c2f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc19ea60 con 0x7f0efc104380 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0f01c2f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc19eba0 con 0x7f0efc101a50 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0f01c2f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc19ed30 con 0x7f0efc106cb0 2024-05-08T22:35:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0efaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc104380 0x7f0efc197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0efaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc104380 0x7f0efc197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60508/0 (socket says 172.21.15.62:60508) 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0efaffd700 1 -- 172.21.15.62:0/214340600 learned_addr learned my addr 172.21.15.62:0/214340600 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0efb7fe700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0efc101a50 0x7f0efc196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.812+0000 7f0efbfff700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0efc106cb0 0x7f0efc19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.813+0000 7f0efaffd700 1 -- 172.21.15.62:0/214340600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0efc106cb0 msgr2=0x7f0efc19e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.813+0000 7f0efaffd700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0efc106cb0 0x7f0efc19e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.813+0000 7f0efaffd700 1 -- 172.21.15.62:0/214340600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0efc101a50 msgr2=0x7f0efc196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.813+0000 7f0efaffd700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0efc101a50 0x7f0efc196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.813+0000 7f0efaffd700 1 -- 172.21.15.62:0/214340600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec0095f0 con 0x7f0efc104380 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.813+0000 7f0efaffd700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc104380 0x7f0efc197460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400d9b0 tx=0x7f0ee400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.813+0000 7f0ef8ff9700 1 -- 172.21.15.62:0/214340600 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4004a30 con 0x7f0efc104380 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.813+0000 7f0ef8ff9700 1 -- 172.21.15.62:0/214340600 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee40054e0 con 0x7f0efc104380 2024-05-08T22:35:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.813+0000 7f0f01c2f700 1 -- 172.21.15.62:0/214340600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0efc19efb0 con 0x7f0efc104380 2024-05-08T22:35:19.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.813+0000 7f0ef8ff9700 1 -- 172.21.15.62:0/214340600 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4009b20 con 0x7f0efc104380 2024-05-08T22:35:19.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.813+0000 7f0f01c2f700 1 -- 172.21.15.62:0/214340600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0efc19f430 con 0x7f0efc104380 2024-05-08T22:35:19.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.814+0000 7f0f01c2f700 1 -- 172.21.15.62:0/214340600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0efc0fcfe0 con 0x7f0efc104380 2024-05-08T22:35:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.815+0000 7f0ef8ff9700 1 -- 172.21.15.62:0/214340600 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0ee4009c80 con 0x7f0efc104380 2024-05-08T22:35:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.815+0000 7f0ef8ff9700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0ee80a11b0 0x7f0ee80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.816+0000 7f0efb7fe700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0ee80a11b0 0x7f0ee80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.816+0000 7f0ef8ff9700 1 -- 172.21.15.62:0/214340600 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0ee40cc7b0 con 0x7f0efc104380 2024-05-08T22:35:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.816+0000 7f0efb7fe700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0ee80a11b0 0x7f0ee80a3670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00b5c0 tx=0x7f0eec005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:19.820+0000 7f0ef8ff9700 1 -- 172.21.15.62:0/214340600 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0ee4095e90 con 0x7f0efc104380 2024-05-08T22:35:19.823 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:35:19 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-b[83604]: 2024-05-08T22:35:19.345+0000 7f0ce2184700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-08T22:35:19.823 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:35:19 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-b[83604]: 2024-05-08T22:35:19.345+0000 7f0ce2184700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-08T22:35:19.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:19 smithi062 ceph-mon[42571]: pgmap v226: 97 pgs: 97 active+clean; 619 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 12 KiB/s wr, 23 op/s 2024-05-08T22:35:19.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:19 smithi062 ceph-mon[42571]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:19.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:19 smithi062 ceph-mon[42571]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:20.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.109+0000 7f0f01c2f700 1 -- 172.21.15.62:0/214340600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f0efc04ec60 con 0x7f0efc104380 2024-05-08T22:35:20.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.113+0000 7f0ef8ff9700 1 -- 172.21.15.62:0/214340600 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f0ee40170f0 con 0x7f0efc104380 2024-05-08T22:35:20.112 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-05-08T22:35:20.113 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAo/jtmkMCKBhAAgF0hcu9jzkRCJTcFJxhGoQ== 2024-05-08T22:35:20.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 -- 172.21.15.62:0/214340600 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0ee80a11b0 msgr2=0x7f0ee80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0ee80a11b0 0x7f0ee80a3670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00b5c0 tx=0x7f0eec005fd0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 -- 172.21.15.62:0/214340600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc104380 msgr2=0x7f0efc197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:20.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc104380 0x7f0efc197460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400d9b0 tx=0x7f0ee400dcc0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 -- 172.21.15.62:0/214340600 shutdown_connections 2024-05-08T22:35:20.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0efc106cb0 0x7f0efc19e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0ee80a11b0 0x7f0ee80a3670 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0efc101a50 0x7f0efc196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 --2- 172.21.15.62:0/214340600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0efc104380 0x7f0efc197460 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:20.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 -- 172.21.15.62:0/214340600 >> 172.21.15.62:0/214340600 conn(0x7f0efc0fb460 msgr2=0x7f0efc103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:20.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 -- 172.21.15.62:0/214340600 shutdown_connections 2024-05-08T22:35:20.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:20.116+0000 7f0f01c2f700 1 -- 172.21.15.62:0/214340600 wait complete. 2024-05-08T22:35:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:19 smithi079 ceph-mon[46738]: pgmap v226: 97 pgs: 97 active+clean; 619 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 12 KiB/s wr, 23 op/s 2024-05-08T22:35:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:19 smithi079 ceph-mon[46738]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:19 smithi079 ceph-mon[46738]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:20.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:19 smithi066 ceph-mon[46830]: pgmap v226: 97 pgs: 97 active+clean; 619 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 12 KiB/s wr, 23 op/s 2024-05-08T22:35:20.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:19 smithi066 ceph-mon[46830]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:20.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:19 smithi066 ceph-mon[46830]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:20.467 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:35:20.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-08T22:35:20.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T22:35:20.509 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T22:35:20.509 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-08T22:35:20.509 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-08T22:35:20.776 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:21.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:20 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/214340600' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T22:35:21.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:20 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/214340600' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T22:35:21.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:20 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/214340600' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T22:35:21.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:20 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/214340600' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T22:35:21.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:20 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/214340600' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T22:35:21.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:20 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/214340600' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T22:35:21.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.782+0000 7f846c52d700 1 -- 172.21.15.62:0/817658116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8464073af0 msgr2=0x7f846410d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:21.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.782+0000 7f846c52d700 1 --2- 172.21.15.62:0/817658116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8464073af0 0x7f846410d460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8460009990 tx=0x7f8460009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:21.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.782+0000 7f846c52d700 1 -- 172.21.15.62:0/817658116 shutdown_connections 2024-05-08T22:35:21.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.782+0000 7f846c52d700 1 --2- 172.21.15.62:0/817658116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f846410d9a0 0x7f846410fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:21.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.782+0000 7f846c52d700 1 --2- 172.21.15.62:0/817658116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84640731d0 0x7f84640735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:21.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.782+0000 7f846c52d700 1 --2- 172.21.15.62:0/817658116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8464073af0 0x7f846410d460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:21.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.782+0000 7f846c52d700 1 -- 172.21.15.62:0/817658116 >> 172.21.15.62:0/817658116 conn(0x7f8464078100 msgr2=0x7f8464078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:21.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.783+0000 7f846c52d700 1 -- 172.21.15.62:0/817658116 shutdown_connections 2024-05-08T22:35:21.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.783+0000 7f846c52d700 1 -- 172.21.15.62:0/817658116 wait complete. 2024-05-08T22:35:21.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.783+0000 7f846c52d700 1 Processor -- start 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.783+0000 7f846c52d700 1 -- start start 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.783+0000 7f846c52d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84640731d0 0x7f8464199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.783+0000 7f846c52d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8464073af0 0x7f8464199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.783+0000 7f846c52d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f846410d9a0 0x7f846419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.783+0000 7f846c52d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846419d580 con 0x7f8464073af0 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.783+0000 7f846c52d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846419d6f0 con 0x7f846410d9a0 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.783+0000 7f846c52d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846419d8b0 con 0x7f84640731d0 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f8469ac8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8464073af0 0x7f8464199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f8469ac8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8464073af0 0x7f8464199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60524/0 (socket says 172.21.15.62:60524) 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f8469ac8700 1 -- 172.21.15.62:0/3451198090 learned_addr learned my addr 172.21.15.62:0/3451198090 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f846aaca700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f846410d9a0 0x7f846419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f846a2c9700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84640731d0 0x7f8464199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f8469ac8700 1 -- 172.21.15.62:0/3451198090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84640731d0 msgr2=0x7f8464199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f8469ac8700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84640731d0 0x7f8464199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f8469ac8700 1 -- 172.21.15.62:0/3451198090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f846410d9a0 msgr2=0x7f846419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f8469ac8700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f846410d9a0 0x7f846419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f8469ac8700 1 -- 172.21.15.62:0/3451198090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84600095f0 con 0x7f8464073af0 2024-05-08T22:35:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f846aaca700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f846410d9a0 0x7f846419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f8469ac8700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8464073af0 0x7f8464199c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f84600098e0 tx=0x7f846000f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f845b7fe700 1 -- 172.21.15.62:0/3451198090 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846001d070 con 0x7f8464073af0 2024-05-08T22:35:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f846c52d700 1 -- 172.21.15.62:0/3451198090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846419db30 con 0x7f8464073af0 2024-05-08T22:35:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f845b7fe700 1 -- 172.21.15.62:0/3451198090 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f846000fb90 con 0x7f8464073af0 2024-05-08T22:35:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f845b7fe700 1 -- 172.21.15.62:0/3451198090 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460017650 con 0x7f8464073af0 2024-05-08T22:35:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.784+0000 7f846c52d700 1 -- 172.21.15.62:0/3451198090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846419de00 con 0x7f8464073af0 2024-05-08T22:35:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.785+0000 7f846c52d700 1 -- 172.21.15.62:0/3451198090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8464108bb0 con 0x7f8464073af0 2024-05-08T22:35:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.786+0000 7f845b7fe700 1 -- 172.21.15.62:0/3451198090 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f846000fd00 con 0x7f8464073af0 2024-05-08T22:35:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.787+0000 7f845b7fe700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84500a11b0 0x7f84500a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.787+0000 7f846a2c9700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84500a11b0 0x7f84500a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.787+0000 7f845b7fe700 1 -- 172.21.15.62:0/3451198090 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f84600cdda0 con 0x7f8464073af0 2024-05-08T22:35:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.788+0000 7f846a2c9700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84500a11b0 0x7f84500a3670 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8454006fd0 tx=0x7f8454008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:21.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:21.791+0000 7f845b7fe700 1 -- 172.21.15.62:0/3451198090 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8460097610 con 0x7f8464073af0 2024-05-08T22:35:21.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:21 smithi062 ceph-mon[42571]: pgmap v227: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-05-08T22:35:22.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.053+0000 7f846c52d700 1 -- 172.21.15.62:0/3451198090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8464074890 con 0x7f8464073af0 2024-05-08T22:35:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.057+0000 7f845b7fe700 1 -- 172.21.15.62:0/3451198090 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+260285 (secure 0 0 0) 0x7f846009ac30 con 0x7f8464073af0 2024-05-08T22:35:22.062 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 -- 172.21.15.62:0/3451198090 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84500a11b0 msgr2=0x7f84500a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84500a11b0 0x7f84500a3670 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8454006fd0 tx=0x7f8454008040 comp rx=0 tx=0).stop 2024-05-08T22:35:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 -- 172.21.15.62:0/3451198090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8464073af0 msgr2=0x7f8464199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8464073af0 0x7f8464199c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f84600098e0 tx=0x7f846000f720 comp rx=0 tx=0).stop 2024-05-08T22:35:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 -- 172.21.15.62:0/3451198090 shutdown_connections 2024-05-08T22:35:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84500a11b0 0x7f84500a3670 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f846410d9a0 0x7f846419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84640731d0 0x7f8464199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 --2- 172.21.15.62:0/3451198090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8464073af0 0x7f8464199c80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 -- 172.21.15.62:0/3451198090 >> 172.21.15.62:0/3451198090 conn(0x7f8464078100 msgr2=0x7f84640ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 -- 172.21.15.62:0/3451198090 shutdown_connections 2024-05-08T22:35:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:22.060+0000 7f846c52d700 1 -- 172.21.15.62:0/3451198090 wait complete. 2024-05-08T22:35:22.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:21 smithi079 ceph-mon[46738]: pgmap v227: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-05-08T22:35:22.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:21 smithi066 ceph-mon[46830]: pgmap v227: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-05-08T22:35:22.425 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":4071146880},{"type":"v1","addr":"172.21.15.62:6801","nonce":4071146880}]},"active_addr":"172.21.15.62:6801/4071146880","active_change":"2024-05-08T22:30:03.951233+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.62:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":3870206031}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":3990867784}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":1829209995}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":4274876935}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":401260228}]}]} 2024-05-08T22:35:22.430 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-08T22:35:22.430 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-08T22:35:22.430 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T22:35:22.661 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:23.045 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:22 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3451198090' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T22:35:23.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:22 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3451198090' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T22:35:23.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:22 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3451198090' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T22:35:23.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.631+0000 7f1680dee700 1 -- 172.21.15.62:0/2904622566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f167c0ffed0 msgr2=0x7f167c100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:23.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.631+0000 7f1680dee700 1 --2- 172.21.15.62:0/2904622566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f167c0ffed0 0x7f167c100350 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1670009990 tx=0x7f1670009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.631+0000 7f1680dee700 1 -- 172.21.15.62:0/2904622566 shutdown_connections 2024-05-08T22:35:23.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.631+0000 7f1680dee700 1 --2- 172.21.15.62:0/2904622566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f167c100890 0x7f167c10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.631+0000 7f1680dee700 1 --2- 172.21.15.62:0/2904622566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f167c105f00 0x7f167c1062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.631+0000 7f1680dee700 1 --2- 172.21.15.62:0/2904622566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f167c0ffed0 0x7f167c100350 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.631+0000 7f1680dee700 1 -- 172.21.15.62:0/2904622566 >> 172.21.15.62:0/2904622566 conn(0x7f167c075f80 msgr2=0x7f167c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.632+0000 7f1680dee700 1 -- 172.21.15.62:0/2904622566 shutdown_connections 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.632+0000 7f1680dee700 1 -- 172.21.15.62:0/2904622566 wait complete. 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.632+0000 7f1680dee700 1 Processor -- start 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.632+0000 7f1680dee700 1 -- start start 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f1680dee700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f167c0ffed0 0x7f167c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f1680dee700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f167c100890 0x7f167c1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f1680dee700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f167c105f00 0x7f167c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f1680dee700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c19c360 con 0x7f167c0ffed0 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f1680dee700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c19c4d0 con 0x7f167c100890 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f1680dee700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c19c690 con 0x7f167c105f00 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f167a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f167c0ffed0 0x7f167c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f167a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f167c0ffed0 0x7f167c197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60546/0 (socket says 172.21.15.62:60546) 2024-05-08T22:35:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f167a59c700 1 -- 172.21.15.62:0/2751658732 learned_addr learned my addr 172.21.15.62:0/2751658732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f1679d9b700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f167c100890 0x7f167c1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f167a59c700 1 -- 172.21.15.62:0/2751658732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f167c105f00 msgr2=0x7f167c19bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f167ad9d700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f167c105f00 0x7f167c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f167a59c700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f167c105f00 0x7f167c19bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f167a59c700 1 -- 172.21.15.62:0/2751658732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f167c100890 msgr2=0x7f167c1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f167a59c700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f167c100890 0x7f167c1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f167a59c700 1 -- 172.21.15.62:0/2751658732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16700095f0 con 0x7f167c0ffed0 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f1679d9b700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f167c100890 0x7f167c1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f167a59c700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f167c0ffed0 0x7f167c197460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f166c00d950 tx=0x7f166c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f166b7fe700 1 -- 172.21.15.62:0/2751658732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c009b40 con 0x7f167c0ffed0 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f166b7fe700 1 -- 172.21.15.62:0/2751658732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f166c009ca0 con 0x7f167c0ffed0 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f1680dee700 1 -- 172.21.15.62:0/2751658732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167c19c970 con 0x7f167c0ffed0 2024-05-08T22:35:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f166b7fe700 1 -- 172.21.15.62:0/2751658732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c00f620 con 0x7f167c0ffed0 2024-05-08T22:35:23.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.633+0000 7f1680dee700 1 -- 172.21.15.62:0/2751658732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c1a41d0 con 0x7f167c0ffed0 2024-05-08T22:35:23.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.634+0000 7f1680dee700 1 -- 172.21.15.62:0/2751658732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167c1015b0 con 0x7f167c0ffed0 2024-05-08T22:35:23.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.635+0000 7f166b7fe700 1 -- 172.21.15.62:0/2751658732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f166c010460 con 0x7f167c0ffed0 2024-05-08T22:35:23.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.636+0000 7f166b7fe700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f16640a1230 0x7f16640a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:23.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.636+0000 7f166b7fe700 1 -- 172.21.15.62:0/2751658732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f166c0ccc00 con 0x7f167c0ffed0 2024-05-08T22:35:23.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.636+0000 7f1679d9b700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f16640a1230 0x7f16640a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:23.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.637+0000 7f1679d9b700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f16640a1230 0x7f16640a36f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1670006010 tx=0x7f167001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:23.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.639+0000 7f166b7fe700 1 -- 172.21.15.62:0/2751658732 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f166c0963e0 con 0x7f167c0ffed0 2024-05-08T22:35:23.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.842+0000 7f1680dee700 1 -- 172.21.15.62:0/2751658732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f167c10fe30 con 0x7f167c0ffed0 2024-05-08T22:35:23.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.843+0000 7f166b7fe700 1 -- 172.21.15.62:0/2751658732 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22047 (secure 0 0 0) 0x7f166c016090 con 0x7f167c0ffed0 2024-05-08T22:35:23.844 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:23.844 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":79,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:35:10.805199+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.845+0000 7f1680dee700 1 -- 172.21.15.62:0/2751658732 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f16640a1230 msgr2=0x7f16640a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.845+0000 7f1680dee700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f16640a1230 0x7f16640a36f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1670006010 tx=0x7f167001a040 comp rx=0 tx=0).stop 2024-05-08T22:35:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.845+0000 7f1680dee700 1 -- 172.21.15.62:0/2751658732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f167c0ffed0 msgr2=0x7f167c197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.845+0000 7f1680dee700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f167c0ffed0 0x7f167c197460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f166c00d950 tx=0x7f166c00dd10 comp rx=0 tx=0).stop 2024-05-08T22:35:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.845+0000 7f1680dee700 1 -- 172.21.15.62:0/2751658732 shutdown_connections 2024-05-08T22:35:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.845+0000 7f1680dee700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f167c105f00 0x7f167c19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.845+0000 7f1680dee700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f167c0ffed0 0x7f167c197460 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.845+0000 7f1680dee700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f16640a1230 0x7f16640a36f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.845+0000 7f1680dee700 1 --2- 172.21.15.62:0/2751658732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f167c100890 0x7f167c1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.845+0000 7f1680dee700 1 -- 172.21.15.62:0/2751658732 >> 172.21.15.62:0/2751658732 conn(0x7f167c075f80 msgr2=0x7f167c068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:23.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.846+0000 7f1680dee700 1 -- 172.21.15.62:0/2751658732 shutdown_connections 2024-05-08T22:35:23.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:23.846+0000 7f1680dee700 1 -- 172.21.15.62:0/2751658732 wait complete. 2024-05-08T22:35:24.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:23 smithi079 ceph-mon[46738]: pgmap v228: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 28 op/s 2024-05-08T22:35:24.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:23 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2751658732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:24.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:23 smithi062 ceph-mon[42571]: pgmap v228: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 28 op/s 2024-05-08T22:35:24.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:23 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2751658732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:24.225 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-08T22:35:24.226 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T22:35:24.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:23 smithi066 ceph-mon[46830]: pgmap v228: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 28 op/s 2024-05-08T22:35:24.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:23 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2751658732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:24.458 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:25.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.324+0000 7f8f0a3db700 1 -- 172.21.15.62:0/54689920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f040ffed0 msgr2=0x7f8f04100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.324+0000 7f8f0a3db700 1 --2- 172.21.15.62:0/54689920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f040ffed0 0x7f8f04100350 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009990 tx=0x7f8ef8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.324+0000 7f8f0a3db700 1 -- 172.21.15.62:0/54689920 shutdown_connections 2024-05-08T22:35:25.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.324+0000 7f8f0a3db700 1 --2- 172.21.15.62:0/54689920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04100890 0x7f8f0410cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.324+0000 7f8f0a3db700 1 --2- 172.21.15.62:0/54689920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f04105f00 0x7f8f041062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.324+0000 7f8f0a3db700 1 --2- 172.21.15.62:0/54689920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f040ffed0 0x7f8f04100350 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.324+0000 7f8f0a3db700 1 -- 172.21.15.62:0/54689920 >> 172.21.15.62:0/54689920 conn(0x7f8f04075f80 msgr2=0x7f8f04076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:25.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.324+0000 7f8f0a3db700 1 -- 172.21.15.62:0/54689920 shutdown_connections 2024-05-08T22:35:25.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.324+0000 7f8f0a3db700 1 -- 172.21.15.62:0/54689920 wait complete. 2024-05-08T22:35:25.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.324+0000 7f8f0a3db700 1 Processor -- start 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f0a3db700 1 -- start start 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f0a3db700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f040ffed0 0x7f8f04101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f0a3db700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f04100890 0x7f8f041020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f0a3db700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04105f00 0x7f8f041025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f0a3db700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f041a5880 con 0x7f8f040ffed0 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f0a3db700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f041a59f0 con 0x7f8f04100890 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f0a3db700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f041a5bb0 con 0x7f8f04105f00 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f03fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f040ffed0 0x7f8f04101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f03fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f040ffed0 0x7f8f04101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60562/0 (socket says 172.21.15.62:60562) 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f03fff700 1 -- 172.21.15.62:0/3960805928 learned_addr learned my addr 172.21.15.62:0/3960805928 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f037fe700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f04100890 0x7f8f041020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f08978700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04105f00 0x7f8f041025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f03fff700 1 -- 172.21.15.62:0/3960805928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04105f00 msgr2=0x7f8f041025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f03fff700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04105f00 0x7f8f041025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f03fff700 1 -- 172.21.15.62:0/3960805928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f04100890 msgr2=0x7f8f041020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f03fff700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f04100890 0x7f8f041020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.325+0000 7f8f03fff700 1 -- 172.21.15.62:0/3960805928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80095f0 con 0x7f8f040ffed0 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.326+0000 7f8f03fff700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f040ffed0 0x7f8f04101b70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400b530 tx=0x7f8ef400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.326+0000 7f8f017fa700 1 -- 172.21.15.62:0/3960805928 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4011830 con 0x7f8f040ffed0 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.326+0000 7f8f017fa700 1 -- 172.21.15.62:0/3960805928 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ef4011e50 con 0x7f8f040ffed0 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.326+0000 7f8f0a3db700 1 -- 172.21.15.62:0/3960805928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f041a5e90 con 0x7f8f040ffed0 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.326+0000 7f8f017fa700 1 -- 172.21.15.62:0/3960805928 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400f2e0 con 0x7f8f040ffed0 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.326+0000 7f8f0a3db700 1 -- 172.21.15.62:0/3960805928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f041a6250 con 0x7f8f040ffed0 2024-05-08T22:35:25.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.327+0000 7f8f0a3db700 1 -- 172.21.15.62:0/3960805928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f041015b0 con 0x7f8f040ffed0 2024-05-08T22:35:25.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.328+0000 7f8f017fa700 1 -- 172.21.15.62:0/3960805928 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8ef4010380 con 0x7f8f040ffed0 2024-05-08T22:35:25.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.328+0000 7f8f017fa700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8eec0a11b0 0x7f8eec0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:25.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.329+0000 7f8f037fe700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8eec0a11b0 0x7f8eec0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:25.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.329+0000 7f8f017fa700 1 -- 172.21.15.62:0/3960805928 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8ef40cc2c0 con 0x7f8f040ffed0 2024-05-08T22:35:25.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.329+0000 7f8f037fe700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8eec0a11b0 0x7f8eec0a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800b5c0 tx=0x7f8ef8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:25.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.333+0000 7f8f017fa700 1 -- 172.21.15.62:0/3960805928 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8ef4095bb0 con 0x7f8f040ffed0 2024-05-08T22:35:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.536+0000 7f8f0a3db700 1 -- 172.21.15.62:0/3960805928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f0404ec60 con 0x7f8f040ffed0 2024-05-08T22:35:25.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.538+0000 7f8f017fa700 1 -- 172.21.15.62:0/3960805928 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22047 (secure 0 0 0) 0x7f8ef40b7e60 con 0x7f8f040ffed0 2024-05-08T22:35:25.538 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:25.538 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":79,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:35:10.805199+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 -- 172.21.15.62:0/3960805928 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8eec0a11b0 msgr2=0x7f8eec0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8eec0a11b0 0x7f8eec0a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800b5c0 tx=0x7f8ef8005960 comp rx=0 tx=0).stop 2024-05-08T22:35:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 -- 172.21.15.62:0/3960805928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f040ffed0 msgr2=0x7f8f04101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f040ffed0 0x7f8f04101b70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400b530 tx=0x7f8ef400b840 comp rx=0 tx=0).stop 2024-05-08T22:35:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 -- 172.21.15.62:0/3960805928 shutdown_connections 2024-05-08T22:35:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f04105f00 0x7f8f041025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f040ffed0 0x7f8f04101b70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f8eec0a11b0 0x7f8eec0a3670 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 --2- 172.21.15.62:0/3960805928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f04100890 0x7f8f041020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 -- 172.21.15.62:0/3960805928 >> 172.21.15.62:0/3960805928 conn(0x7f8f04075f80 msgr2=0x7f8f04068b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 -- 172.21.15.62:0/3960805928 shutdown_connections 2024-05-08T22:35:25.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:25.540+0000 7f8f0a3db700 1 -- 172.21.15.62:0/3960805928 wait complete. 2024-05-08T22:35:25.910 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-08T22:35:25.910 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-08T22:35:25.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-08T22:35:25.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-08T22:35:25.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-08T22:35:25.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-08T22:35:25.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-08T22:35:25.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-08T22:35:25.913 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-08T22:35:25.913 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-08T22:35:25.914 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-08T22:35:25.914 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-08T22:35:26.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:25 smithi062 ceph-mon[42571]: pgmap v229: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-05-08T22:35:26.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:25 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3960805928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:26.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:25 smithi079 ceph-mon[46738]: pgmap v229: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-05-08T22:35:26.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:25 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3960805928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:26.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:25 smithi066 ceph-mon[46830]: pgmap v229: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-05-08T22:35:26.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:25 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3960805928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:27.115 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:27.419 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:27.468 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:28.024 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:28.024 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:28.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:27 smithi062 ceph-mon[42571]: pgmap v230: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-05-08T22:35:28.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:27 smithi079 ceph-mon[46738]: pgmap v230: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-05-08T22:35:28.275 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:28.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:27 smithi066 ceph-mon[46830]: pgmap v230: 97 pgs: 97 active+clean; 626 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-05-08T22:35:29.531 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:29.532 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:29.534 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.137+0000 7f9d04e59700 1 -- 172.21.15.62:0/3912703289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d001019e0 msgr2=0x7f9d0010ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.137+0000 7f9d04e59700 1 --2- 172.21.15.62:0/3912703289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d001019e0 0x7f9d0010ddf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800b0a0 tx=0x7f9cf800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.137+0000 7f9d04e59700 1 -- 172.21.15.62:0/3912703289 shutdown_connections 2024-05-08T22:35:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.137+0000 7f9d04e59700 1 --2- 172.21.15.62:0/3912703289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d001019e0 0x7f9d0010ddf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.137+0000 7f9d04e59700 1 --2- 172.21.15.62:0/3912703289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d00107040 0x7f9d00107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.137+0000 7f9d04e59700 1 --2- 172.21.15.62:0/3912703289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d00101020 0x7f9d001014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.137+0000 7f9d04e59700 1 -- 172.21.15.62:0/3912703289 >> 172.21.15.62:0/3912703289 conn(0x7f9d00078100 msgr2=0x7f9d00078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.137+0000 7f9d04e59700 1 -- 172.21.15.62:0/3912703289 shutdown_connections 2024-05-08T22:35:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.137+0000 7f9d04e59700 1 -- 172.21.15.62:0/3912703289 wait complete. 2024-05-08T22:35:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9d04e59700 1 Processor -- start 2024-05-08T22:35:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9d04e59700 1 -- start start 2024-05-08T22:35:30.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:29 smithi079 ceph-mon[46738]: pgmap v231: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 22 op/s 2024-05-08T22:35:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9d04e59700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d00101020 0x7f9d000725e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9d04e59700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d001019e0 0x7f9d00072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9d04e59700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d00107040 0x7f9d0006b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9d04e59700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0006bfe0 con 0x7f9d001019e0 2024-05-08T22:35:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9d04e59700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0006c150 con 0x7f9d00101020 2024-05-08T22:35:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9d04e59700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0006c310 con 0x7f9d00107040 2024-05-08T22:35:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9cfe59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d00101020 0x7f9d000725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9cfdd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d001019e0 0x7f9d00072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9cfed9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d00107040 0x7f9d0006b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9cfe59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d00101020 0x7f9d000725e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.62:53250/0 (socket says 172.21.15.62:53250) 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9cfed9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d00107040 0x7f9d0006b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.62:60562/0 (socket says 172.21.15.62:60562) 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9cfdd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d001019e0 0x7f9d00072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57206/0 (socket says 172.21.15.62:57206) 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.138+0000 7f9cfed9d700 1 -- 172.21.15.62:0/2777917490 learned_addr learned my addr 172.21.15.62:0/2777917490 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.139+0000 7f9cfed9d700 1 -- 172.21.15.62:0/2777917490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d00101020 msgr2=0x7f9d000725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.139+0000 7f9cfed9d700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d00101020 0x7f9d000725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.139+0000 7f9cfed9d700 1 -- 172.21.15.62:0/2777917490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d001019e0 msgr2=0x7f9d00072b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.139+0000 7f9cfed9d700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d001019e0 0x7f9d00072b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.139+0000 7f9cfed9d700 1 -- 172.21.15.62:0/2777917490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8009d00 con 0x7f9d00107040 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.139+0000 7f9cfe59c700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d00101020 0x7f9d000725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.139+0000 7f9cfdd9b700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d001019e0 0x7f9d00072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.139+0000 7f9cfed9d700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d00107040 0x7f9d0006b810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800b830 tx=0x7f9cf80077f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.142+0000 7f9cef7fe700 1 -- 172.21.15.62:0/2777917490 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800e040 con 0x7f9d00107040 2024-05-08T22:35:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.142+0000 7f9cef7fe700 1 -- 172.21.15.62:0/2777917490 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cf8004c60 con 0x7f9d00107040 2024-05-08T22:35:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.142+0000 7f9d04e59700 1 -- 172.21.15.62:0/2777917490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0006c590 con 0x7f9d00107040 2024-05-08T22:35:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.142+0000 7f9d04e59700 1 -- 172.21.15.62:0/2777917490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d000704f0 con 0x7f9d00107040 2024-05-08T22:35:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.142+0000 7f9cef7fe700 1 -- 172.21.15.62:0/2777917490 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8005280 con 0x7f9d00107040 2024-05-08T22:35:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.142+0000 7f9d04e59700 1 -- 172.21.15.62:0/2777917490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d00111150 con 0x7f9d00107040 2024-05-08T22:35:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.144+0000 7f9cef7fe700 1 -- 172.21.15.62:0/2777917490 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9cf80053e0 con 0x7f9d00107040 2024-05-08T22:35:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.145+0000 7f9cef7fe700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9ce80a1300 0x7f9ce80a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.145+0000 7f9cef7fe700 1 -- 172.21.15.62:0/2777917490 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9cf80ce850 con 0x7f9d00107040 2024-05-08T22:35:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.145+0000 7f9cef7fe700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413] conn(0x7f9ce80aaae0 0x7f9ce80acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.145+0000 7f9cef7fe700 1 -- 172.21.15.62:0/2777917490 --> [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ce80ad5b0 con 0x7f9ce80aaae0 2024-05-08T22:35:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.145+0000 7f9cef7fe700 1 -- 172.21.15.62:0/2777917490 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf80cec00 con 0x7f9d00107040 2024-05-08T22:35:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.145+0000 7f9cfe59c700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9ce80a1300 0x7f9ce80a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.145+0000 7f9cfe59c700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9ce80a1300 0x7f9ce80a37c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0007570 tx=0x7f9cf0010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.146+0000 7f9cfdd9b700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413] conn(0x7f9ce80aaae0 0x7f9ce80acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.146+0000 7f9cfdd9b700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413] conn(0x7f9ce80aaae0 0x7f9ce80acf00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.147+0000 7f9cef7fe700 1 -- 172.21.15.62:0/2777917490 <== osd.10 v2:172.21.15.66:6824/1104097413 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9ce80ad5b0 con 0x7f9ce80aaae0 2024-05-08T22:35:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.163+0000 7f9d04e59700 1 -- 172.21.15.62:0/2777917490 --> [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d00102680 con 0x7f9ce80aaae0 2024-05-08T22:35:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.164+0000 7f9cef7fe700 1 -- 172.21.15.62:0/2777917490 <== osd.10 v2:172.21.15.66:6824/1104097413 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9d00102680 con 0x7f9ce80aaae0 2024-05-08T22:35:30.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.164+0000 7f9d04e59700 1 -- 172.21.15.62:0/2777917490 >> [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413] conn(0x7f9ce80aaae0 msgr2=0x7f9ce80acf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.164+0000 7f9d04e59700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413] conn(0x7f9ce80aaae0 0x7f9ce80acf00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.164+0000 7f9d04e59700 1 -- 172.21.15.62:0/2777917490 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9ce80a1300 msgr2=0x7f9ce80a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.164+0000 7f9d04e59700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9ce80a1300 0x7f9ce80a37c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0007570 tx=0x7f9cf0010040 comp rx=0 tx=0).stop 2024-05-08T22:35:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.164+0000 7f9d04e59700 1 -- 172.21.15.62:0/2777917490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d00107040 msgr2=0x7f9d0006b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.164+0000 7f9d04e59700 1 --2- 172.21.15.62:0/2777917490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d00107040 0x7f9d0006b810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800b830 tx=0x7f9cf80077f0 comp rx=0 tx=0).stop 2024-05-08T22:35:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.164+0000 7f9cfed9d700 1 -- 172.21.15.62:0/2777917490 reap_dead start 2024-05-08T22:35:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.164+0000 7f9d04e59700 1 -- 172.21.15.62:0/2777917490 shutdown_connections 2024-05-08T22:35:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.164+0000 7f9d04e59700 1 -- 172.21.15.62:0/2777917490 >> 172.21.15.62:0/2777917490 conn(0x7f9d00078100 msgr2=0x7f9d000ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.164+0000 7f9d04e59700 1 -- 172.21.15.62:0/2777917490 shutdown_connections 2024-05-08T22:35:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:30.165+0000 7f9d04e59700 1 -- 172.21.15.62:0/2777917490 wait complete. 2024-05-08T22:35:30.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:29 smithi062 ceph-mon[42571]: pgmap v231: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 22 op/s 2024-05-08T22:35:30.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:29 smithi066 ceph-mon[46830]: pgmap v231: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 22 op/s 2024-05-08T22:35:31.171 INFO:teuthology.orchestra.run.smithi062.stdout:287762808850 2024-05-08T22:35:31.171 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-08T22:35:31.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.549+0000 7f824fcac700 1 -- 172.21.15.62:0/1587750102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82480691e0 msgr2=0x7f82480695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.549+0000 7f824fcac700 1 --2- 172.21.15.62:0/1587750102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82480691e0 0x7f82480695c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8244009990 tx=0x7f8244009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.549+0000 7f824fcac700 1 -- 172.21.15.62:0/1587750102 shutdown_connections 2024-05-08T22:35:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.549+0000 7f824fcac700 1 --2- 172.21.15.62:0/1587750102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f824810da40 0x7f824810fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.549+0000 7f824fcac700 1 --2- 172.21.15.62:0/1587750102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82480691e0 0x7f82480695c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.549+0000 7f824fcac700 1 --2- 172.21.15.62:0/1587750102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8248069b00 0x7f824810d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.549+0000 7f824fcac700 1 -- 172.21.15.62:0/1587750102 >> 172.21.15.62:0/1587750102 conn(0x7f8248076e70 msgr2=0x7f8248077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.550+0000 7f824fcac700 1 -- 172.21.15.62:0/1587750102 shutdown_connections 2024-05-08T22:35:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.550+0000 7f824fcac700 1 -- 172.21.15.62:0/1587750102 wait complete. 2024-05-08T22:35:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.550+0000 7f824fcac700 1 Processor -- start 2024-05-08T22:35:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.550+0000 7f824fcac700 1 -- start start 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.550+0000 7f824fcac700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82480691e0 0x7f82481995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.550+0000 7f824fcac700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8248069b00 0x7f8248199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.550+0000 7f824fcac700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f824810da40 0x7f824819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.550+0000 7f824fcac700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824819e4a0 con 0x7f82480691e0 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824fcac700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824819e610 con 0x7f824810da40 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824fcac700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824819e7d0 con 0x7f8248069b00 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824da48700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82480691e0 0x7f82481995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824da48700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82480691e0 0x7f82481995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57230/0 (socket says 172.21.15.62:57230) 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824da48700 1 -- 172.21.15.62:0/1248807248 learned_addr learned my addr 172.21.15.62:0/1248807248 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824d247700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8248069b00 0x7f8248199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824e249700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f824810da40 0x7f824819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824da48700 1 -- 172.21.15.62:0/1248807248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8248069b00 msgr2=0x7f8248199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824da48700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8248069b00 0x7f8248199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824da48700 1 -- 172.21.15.62:0/1248807248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f824810da40 msgr2=0x7f824819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824da48700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f824810da40 0x7f824819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824da48700 1 -- 172.21.15.62:0/1248807248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440095f0 con 0x7f82480691e0 2024-05-08T22:35:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824d247700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8248069b00 0x7f8248199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824e249700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f824810da40 0x7f824819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824da48700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82480691e0 0x7f82481995a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82440098e0 tx=0x7f824400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f823effd700 1 -- 172.21.15.62:0/1248807248 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824401d070 con 0x7f82480691e0 2024-05-08T22:35:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f823effd700 1 -- 172.21.15.62:0/1248807248 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8244005590 con 0x7f82480691e0 2024-05-08T22:35:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824fcac700 1 -- 172.21.15.62:0/1248807248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824819ea50 con 0x7f82480691e0 2024-05-08T22:35:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f823effd700 1 -- 172.21.15.62:0/1248807248 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244003de0 con 0x7f82480691e0 2024-05-08T22:35:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.551+0000 7f824fcac700 1 -- 172.21.15.62:0/1248807248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82481a6310 con 0x7f82480691e0 2024-05-08T22:35:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.552+0000 7f824fcac700 1 -- 172.21.15.62:0/1248807248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8248113390 con 0x7f82480691e0 2024-05-08T22:35:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.554+0000 7f823effd700 1 -- 172.21.15.62:0/1248807248 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f824400fc40 con 0x7f82480691e0 2024-05-08T22:35:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.554+0000 7f823effd700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f82340a11b0 0x7f82340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.554+0000 7f823effd700 1 -- 172.21.15.62:0/1248807248 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f82440cdc00 con 0x7f82480691e0 2024-05-08T22:35:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.554+0000 7f824d247700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f82340a11b0 0x7f82340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.555+0000 7f823effd700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040] conn(0x7f82340aa990 0x7f82340acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.555+0000 7f823effd700 1 -- 172.21.15.62:0/1248807248 --> [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82340ad460 con 0x7f82340aa990 2024-05-08T22:35:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.555+0000 7f823effd700 1 -- 172.21.15.62:0/1248807248 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82440ce0a0 con 0x7f82480691e0 2024-05-08T22:35:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.555+0000 7f824d247700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f82340a11b0 0x7f82340a3670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8238005fd0 tx=0x7f8238005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.555+0000 7f824e249700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040] conn(0x7f82340aa990 0x7f82340acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.555+0000 7f824e249700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040] conn(0x7f82340aa990 0x7f82340acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.556+0000 7f823effd700 1 -- 172.21.15.62:0/1248807248 <== osd.8 v2:172.21.15.79:6816/2871988040 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f82340ad460 con 0x7f82340aa990 2024-05-08T22:35:31.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.573+0000 7f824fcac700 1 -- 172.21.15.62:0/1248807248 --> [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f824819ed20 con 0x7f82340aa990 2024-05-08T22:35:31.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f823effd700 1 -- 172.21.15.62:0/1248807248 <== osd.8 v2:172.21.15.79:6816/2871988040 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8240009c60 con 0x7f82340aa990 2024-05-08T22:35:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f824fcac700 1 -- 172.21.15.62:0/1248807248 >> [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040] conn(0x7f82340aa990 msgr2=0x7f82340acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f824fcac700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040] conn(0x7f82340aa990 0x7f82340acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f824fcac700 1 -- 172.21.15.62:0/1248807248 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f82340a11b0 msgr2=0x7f82340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f824fcac700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f82340a11b0 0x7f82340a3670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8238005fd0 tx=0x7f8238005dc0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f824fcac700 1 -- 172.21.15.62:0/1248807248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82480691e0 msgr2=0x7f82481995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f824fcac700 1 --2- 172.21.15.62:0/1248807248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82480691e0 0x7f82481995a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82440098e0 tx=0x7f824400f740 comp rx=0 tx=0).stop 2024-05-08T22:35:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f824e249700 1 -- 172.21.15.62:0/1248807248 reap_dead start 2024-05-08T22:35:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f824fcac700 1 -- 172.21.15.62:0/1248807248 shutdown_connections 2024-05-08T22:35:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f824fcac700 1 -- 172.21.15.62:0/1248807248 >> 172.21.15.62:0/1248807248 conn(0x7f8248076e70 msgr2=0x7f824810f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f824fcac700 1 -- 172.21.15.62:0/1248807248 shutdown_connections 2024-05-08T22:35:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.574+0000 7f824fcac700 1 -- 172.21.15.62:0/1248807248 wait complete. 2024-05-08T22:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.609+0000 7fbca21d2700 1 -- 172.21.15.62:0/3100654088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc9c069bd0 msgr2=0x7fbc9c10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.609+0000 7fbca21d2700 1 --2- 172.21.15.62:0/3100654088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc9c069bd0 0x7fbc9c10de00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400b0d0 tx=0x7fbc9400b3e0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.609+0000 7fbca21d2700 1 -- 172.21.15.62:0/3100654088 shutdown_connections 2024-05-08T22:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.609+0000 7fbca21d2700 1 --2- 172.21.15.62:0/3100654088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc9c069bd0 0x7fbc9c10de00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.609+0000 7fbca21d2700 1 --2- 172.21.15.62:0/3100654088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc9c106fe0 0x7fbc9c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.609+0000 7fbca21d2700 1 --2- 172.21.15.62:0/3100654088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc9c069210 0x7fbc9c069690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.609+0000 7fbca21d2700 1 -- 172.21.15.62:0/3100654088 >> 172.21.15.62:0/3100654088 conn(0x7fbc9c076e70 msgr2=0x7fbc9c077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.609+0000 7fbca21d2700 1 -- 172.21.15.62:0/3100654088 shutdown_connections 2024-05-08T22:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.609+0000 7fbca21d2700 1 -- 172.21.15.62:0/3100654088 wait complete. 2024-05-08T22:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.610+0000 7fbca21d2700 1 Processor -- start 2024-05-08T22:35:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.610+0000 7fbca21d2700 1 -- start start 2024-05-08T22:35:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.610+0000 7fbca21d2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc9c069210 0x7fbc9c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.610+0000 7fbca21d2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc9c069bd0 0x7fbc9c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.610+0000 7fbca21d2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc9c106fe0 0x7fbc9c19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.610+0000 7fbca21d2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c19c390 con 0x7fbc9c106fe0 2024-05-08T22:35:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.610+0000 7fbca21d2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c19c500 con 0x7fbc9c069bd0 2024-05-08T22:35:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.610+0000 7fbca21d2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c19c6c0 con 0x7fbc9c069210 2024-05-08T22:35:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc9bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc9c106fe0 0x7fbc9c19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc9bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc9c106fe0 0x7fbc9c19bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57248/0 (socket says 172.21.15.62:57248) 2024-05-08T22:35:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc9bfff700 1 -- 172.21.15.62:0/3961048799 learned_addr learned my addr 172.21.15.62:0/3961048799 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc9b7fe700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc9c069210 0x7fbc9c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc9affd700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc9c069bd0 0x7fbc9c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc9bfff700 1 -- 172.21.15.62:0/3961048799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc9c069210 msgr2=0x7fbc9c1974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc9bfff700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc9c069210 0x7fbc9c1974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc9bfff700 1 -- 172.21.15.62:0/3961048799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc9c069bd0 msgr2=0x7fbc9c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc9bfff700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc9c069bd0 0x7fbc9c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc9bfff700 1 -- 172.21.15.62:0/3961048799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc94009d00 con 0x7fbc9c106fe0 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc9bfff700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc9c106fe0 0x7fbc9c19bd80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400bd50 tx=0x7fbc9400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc98ff9700 1 -- 172.21.15.62:0/3961048799 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9400e040 con 0x7fbc9c106fe0 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc98ff9700 1 -- 172.21.15.62:0/3961048799 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9401b8e0 con 0x7fbc9c106fe0 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbc98ff9700 1 -- 172.21.15.62:0/3961048799 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc940128f0 con 0x7fbc9c106fe0 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbca21d2700 1 -- 172.21.15.62:0/3961048799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9c19c940 con 0x7fbc9c106fe0 2024-05-08T22:35:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.611+0000 7fbca21d2700 1 -- 172.21.15.62:0/3961048799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9c1a4200 con 0x7fbc9c106fe0 2024-05-08T22:35:31.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.612+0000 7fbca21d2700 1 -- 172.21.15.62:0/3961048799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbc9c111150 con 0x7fbc9c106fe0 2024-05-08T22:35:31.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.613+0000 7fbc98ff9700 1 -- 172.21.15.62:0/3961048799 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbc9401ba50 con 0x7fbc9c106fe0 2024-05-08T22:35:31.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.614+0000 7fbc98ff9700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbc840a1230 0x7fbc840a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.614+0000 7fbc98ff9700 1 -- 172.21.15.62:0/3961048799 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbc940cdfd0 con 0x7fbc9c106fe0 2024-05-08T22:35:31.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.614+0000 7fbc9b7fe700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbc840a1230 0x7fbc840a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.614+0000 7fbc98ff9700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145] conn(0x7fbc840aaa10 0x7fbc840ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:31.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.614+0000 7fbc98ff9700 1 -- 172.21.15.62:0/3961048799 --> [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc840ad4e0 con 0x7fbc840aaa10 2024-05-08T22:35:31.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.614+0000 7fbc98ff9700 1 -- 172.21.15.62:0/3961048799 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc940ce470 con 0x7fbc9c106fe0 2024-05-08T22:35:31.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.615+0000 7fbc9affd700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145] conn(0x7fbc840aaa10 0x7fbc840ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:31.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.615+0000 7fbc9b7fe700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbc840a1230 0x7fbc840a36f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c005950 tx=0x7fbc8c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.615+0000 7fbc9affd700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145] conn(0x7fbc840aaa10 0x7fbc840ace30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:31.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.616+0000 7fbc98ff9700 1 -- 172.21.15.62:0/3961048799 <== osd.6 v2:172.21.15.62:6818/1217656145 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fbc840ad4e0 con 0x7fbc840aaa10 2024-05-08T22:35:31.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.633+0000 7fbca21d2700 1 -- 172.21.15.62:0/3961048799 --> [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbc9c102690 con 0x7fbc840aaa10 2024-05-08T22:35:31.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.633+0000 7fbc98ff9700 1 -- 172.21.15.62:0/3961048799 <== osd.6 v2:172.21.15.62:6818/1217656145 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbc9c102690 con 0x7fbc840aaa10 2024-05-08T22:35:31.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.634+0000 7fbca21d2700 1 -- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145] conn(0x7fbc840aaa10 msgr2=0x7fbc840ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.634+0000 7fbca21d2700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145] conn(0x7fbc840aaa10 0x7fbc840ace30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:31.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.634+0000 7fbca21d2700 1 -- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbc840a1230 msgr2=0x7fbc840a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.634+0000 7fbca21d2700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbc840a1230 0x7fbc840a36f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c005950 tx=0x7fbc8c00b410 comp rx=0 tx=0).stop 2024-05-08T22:35:31.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.634+0000 7fbca21d2700 1 -- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc9c106fe0 msgr2=0x7fbc9c19bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:31.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.634+0000 7fbca21d2700 1 --2- 172.21.15.62:0/3961048799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc9c106fe0 0x7fbc9c19bd80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400bd50 tx=0x7fbc9400bd80 comp rx=0 tx=0).stop 2024-05-08T22:35:31.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.634+0000 7fbc9bfff700 1 -- 172.21.15.62:0/3961048799 reap_dead start 2024-05-08T22:35:31.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.634+0000 7fbca21d2700 1 -- 172.21.15.62:0/3961048799 shutdown_connections 2024-05-08T22:35:31.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.634+0000 7fbca21d2700 1 -- 172.21.15.62:0/3961048799 >> 172.21.15.62:0/3961048799 conn(0x7fbc9c076e70 msgr2=0x7fbc9c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:31.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.634+0000 7fbca21d2700 1 -- 172.21.15.62:0/3961048799 shutdown_connections 2024-05-08T22:35:31.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:31.634+0000 7fbca21d2700 1 -- 172.21.15.62:0/3961048799 wait complete. 2024-05-08T22:35:31.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:31 smithi062 ceph-mon[42571]: pgmap v232: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-05-08T22:35:32.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:31 smithi079 ceph-mon[46738]: pgmap v232: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-05-08T22:35:32.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.278+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2668200742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dc010da80 msgr2=0x7f2dc010fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.278+0000 7f2dc53ab700 1 --2- 172.21.15.62:0/2668200742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dc010da80 0x7f2dc010fe70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2db800b0a0 tx=0x7f2db800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.279+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2668200742 shutdown_connections 2024-05-08T22:35:32.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.279+0000 7f2dc53ab700 1 --2- 172.21.15.62:0/2668200742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dc010da80 0x7f2dc010fe70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.279+0000 7f2dc53ab700 1 --2- 172.21.15.62:0/2668200742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc0068950 0x7f2dc0068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.279+0000 7f2dc53ab700 1 --2- 172.21.15.62:0/2668200742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dc0069270 0x7f2dc010d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.279+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2668200742 >> 172.21.15.62:0/2668200742 conn(0x7f2dc0075f80 msgr2=0x7f2dc0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:32.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.279+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2668200742 shutdown_connections 2024-05-08T22:35:32.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.279+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2668200742 wait complete. 2024-05-08T22:35:32.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dc53ab700 1 Processor -- start 2024-05-08T22:35:32.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dc53ab700 1 -- start start 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dc53ab700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dc0068950 0x7f2dc01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dc53ab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dc0069270 0x7f2dc0199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dc53ab700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc010da80 0x7f2dc019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dc53ab700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc019e550 con 0x7f2dc0068950 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dc53ab700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc019e6c0 con 0x7f2dc010da80 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dc53ab700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc019e880 con 0x7f2dc0069270 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dbeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dc0068950 0x7f2dc01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dbeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dc0068950 0x7f2dc01996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57270/0 (socket says 172.21.15.62:57270) 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dbeffd700 1 -- 172.21.15.62:0/2160061238 learned_addr learned my addr 172.21.15.62:0/2160061238 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dbe7fc700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dc0069270 0x7f2dc0199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.280+0000 7f2dbf7fe700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc010da80 0x7f2dc019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.281+0000 7f2dbeffd700 1 -- 172.21.15.62:0/2160061238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dc0069270 msgr2=0x7f2dc0199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.281+0000 7f2dbeffd700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dc0069270 0x7f2dc0199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.281+0000 7f2dbeffd700 1 -- 172.21.15.62:0/2160061238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc010da80 msgr2=0x7f2dc019df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.281+0000 7f2dbeffd700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dc010da80 0x7f2dc019df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.281+0000 7f2dbeffd700 1 -- 172.21.15.62:0/2160061238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db8009d00 con 0x7f2dc0068950 2024-05-08T22:35:32.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.281+0000 7f2dbeffd700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dc0068950 0x7f2dc01996a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2db000e9a0 tx=0x7f2db000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:32.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.281+0000 7f2da7fff700 1 -- 172.21.15.62:0/2160061238 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0009790 con 0x7f2dc0068950 2024-05-08T22:35:32.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.281+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2160061238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc019eb60 con 0x7f2dc0068950 2024-05-08T22:35:32.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.281+0000 7f2da7fff700 1 -- 172.21.15.62:0/2160061238 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db0010430 con 0x7f2dc0068950 2024-05-08T22:35:32.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.281+0000 7f2da7fff700 1 -- 172.21.15.62:0/2160061238 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0018680 con 0x7f2dc0068950 2024-05-08T22:35:32.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.281+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2160061238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc01a63c0 con 0x7f2dc0068950 2024-05-08T22:35:32.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.282+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2160061238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2dc0113350 con 0x7f2dc0068950 2024-05-08T22:35:32.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.283+0000 7f2da7fff700 1 -- 172.21.15.62:0/2160061238 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2db00105a0 con 0x7f2dc0068950 2024-05-08T22:35:32.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.284+0000 7f2da7fff700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2da80a1230 0x7f2da80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.284+0000 7f2dbe7fc700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2da80a1230 0x7f2da80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.284+0000 7f2da7fff700 1 -- 172.21.15.62:0/2160061238 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2db0014070 con 0x7f2dc0068950 2024-05-08T22:35:32.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.284+0000 7f2da7fff700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835] conn(0x7f2da80aaa10 0x7f2da80ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:32.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.284+0000 7f2da7fff700 1 -- 172.21.15.62:0/2160061238 --> [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2da80ad4e0 con 0x7f2da80aaa10 2024-05-08T22:35:32.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.284+0000 7f2da7fff700 1 -- 172.21.15.62:0/2160061238 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2db00cd4e0 con 0x7f2dc0068950 2024-05-08T22:35:32.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.285+0000 7f2dbe7fc700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2da80a1230 0x7f2da80a36f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2dc019abb0 tx=0x7f2db400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:32.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.285+0000 7f2dbf7fe700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835] conn(0x7f2da80aaa10 0x7f2da80ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:32.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.285+0000 7f2dbf7fe700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835] conn(0x7f2da80aaa10 0x7f2da80ace30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:32.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.286+0000 7f2da7fff700 1 -- 172.21.15.62:0/2160061238 <== osd.2 v2:172.21.15.79:6800/2543768835 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2da80ad4e0 con 0x7f2da80aaa10 2024-05-08T22:35:32.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.303+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2160061238 --> [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2dc01004b0 con 0x7f2da80aaa10 2024-05-08T22:35:32.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.303+0000 7f2da7fff700 1 -- 172.21.15.62:0/2160061238 <== osd.2 v2:172.21.15.79:6800/2543768835 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2dc01004b0 con 0x7f2da80aaa10 2024-05-08T22:35:32.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.304+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2160061238 >> [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835] conn(0x7f2da80aaa10 msgr2=0x7f2da80ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.304+0000 7f2dc53ab700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835] conn(0x7f2da80aaa10 0x7f2da80ace30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.304+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2160061238 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2da80a1230 msgr2=0x7f2da80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.304+0000 7f2dc53ab700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f2da80a1230 0x7f2da80a36f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2dc019abb0 tx=0x7f2db400a380 comp rx=0 tx=0).stop 2024-05-08T22:35:32.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.304+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2160061238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dc0068950 msgr2=0x7f2dc01996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:32.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.304+0000 7f2dc53ab700 1 --2- 172.21.15.62:0/2160061238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2dc0068950 0x7f2dc01996a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2db000e9a0 tx=0x7f2db000ecb0 comp rx=0 tx=0).stop 2024-05-08T22:35:32.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.304+0000 7f2dbf7fe700 1 -- 172.21.15.62:0/2160061238 reap_dead start 2024-05-08T22:35:32.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.304+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2160061238 shutdown_connections 2024-05-08T22:35:32.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.304+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2160061238 >> 172.21.15.62:0/2160061238 conn(0x7f2dc0075f80 msgr2=0x7f2dc010eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:32.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.304+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2160061238 shutdown_connections 2024-05-08T22:35:32.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.304+0000 7f2dc53ab700 1 -- 172.21.15.62:0/2160061238 wait complete. 2024-05-08T22:35:32.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:31 smithi066 ceph-mon[46830]: pgmap v232: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-05-08T22:35:32.557 INFO:teuthology.orchestra.run.smithi062.stdout:236223201305 2024-05-08T22:35:32.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-08T22:35:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.190+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/609886860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d98106fb0 msgr2=0x7f1d98107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.190+0000 7f1d9eb2a700 1 --2- 172.21.15.62:0/609886860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d98106fb0 0x7f1d98107390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009990 tx=0x7f1d8c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.190+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/609886860 shutdown_connections 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.190+0000 7f1d9eb2a700 1 --2- 172.21.15.62:0/609886860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d980692e0 0x7f1d9810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.190+0000 7f1d9eb2a700 1 --2- 172.21.15.62:0/609886860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d98106fb0 0x7f1d98107390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.190+0000 7f1d9eb2a700 1 --2- 172.21.15.62:0/609886860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d98068920 0x7f1d98068da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.190+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/609886860 >> 172.21.15.62:0/609886860 conn(0x7f1d98076ee0 msgr2=0x7f1d980772f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.190+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/609886860 shutdown_connections 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.190+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/609886860 wait complete. 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.190+0000 7f1d9eb2a700 1 Processor -- start 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9eb2a700 1 -- start start 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9eb2a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d98068920 0x7f1d980ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9eb2a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d980692e0 0x7f1d98100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9eb2a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d98106fb0 0x7f1d98100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9eb2a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d981a5710 con 0x7f1d98068920 2024-05-08T22:35:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9eb2a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d981a5880 con 0x7f1d98106fb0 2024-05-08T22:35:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9eb2a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d981a5a40 con 0x7f1d980692e0 2024-05-08T22:35:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9c8c6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d98068920 0x7f1d980ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9c8c6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d98068920 0x7f1d980ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57284/0 (socket says 172.21.15.62:57284) 2024-05-08T22:35:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9c8c6700 1 -- 172.21.15.62:0/136536730 learned_addr learned my addr 172.21.15.62:0/136536730 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9d0c7700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d98106fb0 0x7f1d98100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d97fff700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d980692e0 0x7f1d98100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9c8c6700 1 -- 172.21.15.62:0/136536730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d980692e0 msgr2=0x7f1d98100410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9c8c6700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d980692e0 0x7f1d98100410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9c8c6700 1 -- 172.21.15.62:0/136536730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d98106fb0 msgr2=0x7f1d98100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9c8c6700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d98106fb0 0x7f1d98100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9c8c6700 1 -- 172.21.15.62:0/136536730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009520 con 0x7f1d98068920 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.191+0000 7f1d9c8c6700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d98068920 0x7f1d980ffed0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c000c00 tx=0x7f1d8c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.192+0000 7f1d95ffb700 1 -- 172.21.15.62:0/136536730 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c01d070 con 0x7f1d98068920 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.192+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/136536730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d90009c40 con 0x7f1d98068920 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.192+0000 7f1d95ffb700 1 -- 172.21.15.62:0/136536730 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d8c022470 con 0x7f1d98068920 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.192+0000 7f1d95ffb700 1 -- 172.21.15.62:0/136536730 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c017650 con 0x7f1d98068920 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.192+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/136536730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8c0095f0 con 0x7f1d98068920 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.193+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/136536730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d98111160 con 0x7f1d98068920 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.194+0000 7f1d95ffb700 1 -- 172.21.15.62:0/136536730 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1d8c0225e0 con 0x7f1d98068920 2024-05-08T22:35:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.195+0000 7f1d95ffb700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f1d880a11b0 0x7f1d880a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.195+0000 7f1d97fff700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f1d880a11b0 0x7f1d880a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.195+0000 7f1d95ffb700 1 -- 172.21.15.62:0/136536730 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1d8c0cdad0 con 0x7f1d98068920 2024-05-08T22:35:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.195+0000 7f1d95ffb700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866] conn(0x7f1d880aa990 0x7f1d880acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.195+0000 7f1d95ffb700 1 -- 172.21.15.62:0/136536730 --> [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d880ad460 con 0x7f1d880aa990 2024-05-08T22:35:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.195+0000 7f1d95ffb700 1 -- 172.21.15.62:0/136536730 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d8c0d22c0 con 0x7f1d98068920 2024-05-08T22:35:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.195+0000 7f1d97fff700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f1d880a11b0 0x7f1d880a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1d98101600 tx=0x7f1d84009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.195+0000 7f1d9d0c7700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866] conn(0x7f1d880aa990 0x7f1d880acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.196+0000 7f1d9d0c7700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866] conn(0x7f1d880aa990 0x7f1d880acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.197+0000 7f1d95ffb700 1 -- 172.21.15.62:0/136536730 <== osd.4 v2:172.21.15.66:6808/2311123866 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1d880ad460 con 0x7f1d880aa990 2024-05-08T22:35:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.213+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/136536730 --> [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1d98102660 con 0x7f1d880aa990 2024-05-08T22:35:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.213+0000 7f1d95ffb700 1 -- 172.21.15.62:0/136536730 <== osd.4 v2:172.21.15.66:6808/2311123866 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1d98102660 con 0x7f1d880aa990 2024-05-08T22:35:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.214+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/136536730 >> [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866] conn(0x7f1d880aa990 msgr2=0x7f1d880acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.214+0000 7f1d9eb2a700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866] conn(0x7f1d880aa990 0x7f1d880acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.214+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/136536730 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f1d880a11b0 msgr2=0x7f1d880a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.214+0000 7f1d9eb2a700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f1d880a11b0 0x7f1d880a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1d98101600 tx=0x7f1d84009260 comp rx=0 tx=0).stop 2024-05-08T22:35:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.214+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/136536730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d98068920 msgr2=0x7f1d980ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.214+0000 7f1d9eb2a700 1 --2- 172.21.15.62:0/136536730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d98068920 0x7f1d980ffed0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c000c00 tx=0x7f1d8c005740 comp rx=0 tx=0).stop 2024-05-08T22:35:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.214+0000 7f1d9d0c7700 1 -- 172.21.15.62:0/136536730 reap_dead start 2024-05-08T22:35:33.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.214+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/136536730 shutdown_connections 2024-05-08T22:35:33.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.214+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/136536730 >> 172.21.15.62:0/136536730 conn(0x7f1d98076ee0 msgr2=0x7f1d980feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.214+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/136536730 shutdown_connections 2024-05-08T22:35:33.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.214+0000 7f1d9eb2a700 1 -- 172.21.15.62:0/136536730 wait complete. 2024-05-08T22:35:33.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.595+0000 7f03a9ec8700 1 -- 172.21.15.62:0/332567024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a4107020 msgr2=0x7f03a4107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.595+0000 7f03a9ec8700 1 --2- 172.21.15.62:0/332567024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a4107020 0x7f03a4107400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0394009990 tx=0x7f0394009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.595+0000 7f03a9ec8700 1 -- 172.21.15.62:0/332567024 shutdown_connections 2024-05-08T22:35:33.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.595+0000 7f03a9ec8700 1 --2- 172.21.15.62:0/332567024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03a40692f0 0x7f03a410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.595+0000 7f03a9ec8700 1 --2- 172.21.15.62:0/332567024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a4107020 0x7f03a4107400 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.595+0000 7f03a9ec8700 1 --2- 172.21.15.62:0/332567024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03a4068950 0x7f03a4068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.595+0000 7f03a9ec8700 1 -- 172.21.15.62:0/332567024 >> 172.21.15.62:0/332567024 conn(0x7f03a4075f80 msgr2=0x7f03a4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.595+0000 7f03a9ec8700 1 -- 172.21.15.62:0/332567024 shutdown_connections 2024-05-08T22:35:33.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.595+0000 7f03a9ec8700 1 -- 172.21.15.62:0/332567024 wait complete. 2024-05-08T22:35:33.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.595+0000 7f03a9ec8700 1 Processor -- start 2024-05-08T22:35:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a9ec8700 1 -- start start 2024-05-08T22:35:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a9ec8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03a4068950 0x7f03a40fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a9ec8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a40692f0 0x7f03a41004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a9ec8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03a4107020 0x7f03a4100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a9ec8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a41a5730 con 0x7f03a40692f0 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a9ec8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a41a58a0 con 0x7f03a4107020 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a9ec8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a41a5a60 con 0x7f03a4068950 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a40692f0 0x7f03a41004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a40692f0 0x7f03a41004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57320/0 (socket says 172.21.15.62:57320) 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a2ffd700 1 -- 172.21.15.62:0/3886414344 learned_addr learned my addr 172.21.15.62:0/3886414344 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a37fe700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03a4068950 0x7f03a40fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a3fff700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03a4107020 0x7f03a4100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a2ffd700 1 -- 172.21.15.62:0/3886414344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03a4068950 msgr2=0x7f03a40fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a2ffd700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03a4068950 0x7f03a40fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a2ffd700 1 -- 172.21.15.62:0/3886414344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03a4107020 msgr2=0x7f03a4100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a2ffd700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03a4107020 0x7f03a4100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a2ffd700 1 -- 172.21.15.62:0/3886414344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03940095f0 con 0x7f03a40692f0 2024-05-08T22:35:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.596+0000 7f03a2ffd700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a40692f0 0x7f03a41004e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f039800d950 tx=0x7f039800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.597+0000 7f03a0ff9700 1 -- 172.21.15.62:0/3886414344 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039800f890 con 0x7f03a40692f0 2024-05-08T22:35:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.597+0000 7f03a9ec8700 1 -- 172.21.15.62:0/3886414344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a41a5d40 con 0x7f03a40692f0 2024-05-08T22:35:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.597+0000 7f03a0ff9700 1 -- 172.21.15.62:0/3886414344 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0398010460 con 0x7f03a40692f0 2024-05-08T22:35:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.597+0000 7f03a0ff9700 1 -- 172.21.15.62:0/3886414344 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039800e610 con 0x7f03a40692f0 2024-05-08T22:35:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.597+0000 7f03a9ec8700 1 -- 172.21.15.62:0/3886414344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a41a6230 con 0x7f03a40692f0 2024-05-08T22:35:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.597+0000 7f03a9ec8700 1 -- 172.21.15.62:0/3886414344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f03a4111120 con 0x7f03a40692f0 2024-05-08T22:35:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.599+0000 7f03a0ff9700 1 -- 172.21.15.62:0/3886414344 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f03980108d0 con 0x7f03a40692f0 2024-05-08T22:35:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.600+0000 7f03a0ff9700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f038c0a1230 0x7f038c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.600+0000 7f03a37fe700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f038c0a1230 0x7f038c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.600+0000 7f03a0ff9700 1 -- 172.21.15.62:0/3886414344 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f03980ccb00 con 0x7f03a40692f0 2024-05-08T22:35:33.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.600+0000 7f03a0ff9700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230] conn(0x7f038c0aaa10 0x7f038c0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.600+0000 7f03a0ff9700 1 -- 172.21.15.62:0/3886414344 --> [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f038c0ad4e0 con 0x7f038c0aaa10 2024-05-08T22:35:33.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.600+0000 7f03a0ff9700 1 -- 172.21.15.62:0/3886414344 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03980ccfa0 con 0x7f03a40692f0 2024-05-08T22:35:33.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.600+0000 7f03a3fff700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230] conn(0x7f038c0aaa10 0x7f038c0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.600+0000 7f03a37fe700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f038c0a1230 0x7f038c0a36f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f039400b5c0 tx=0x7f0394005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.600+0000 7f03a3fff700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230] conn(0x7f038c0aaa10 0x7f038c0ace30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.601+0000 7f03a0ff9700 1 -- 172.21.15.62:0/3886414344 <== osd.3 v2:172.21.15.62:6810/2546077230 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f038c0ad4e0 con 0x7f038c0aaa10 2024-05-08T22:35:33.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.618+0000 7f03a9ec8700 1 -- 172.21.15.62:0/3886414344 --> [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f03a4061620 con 0x7f038c0aaa10 2024-05-08T22:35:33.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.618+0000 7f03a0ff9700 1 -- 172.21.15.62:0/3886414344 <== osd.3 v2:172.21.15.62:6810/2546077230 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f03a4061620 con 0x7f038c0aaa10 2024-05-08T22:35:33.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.619+0000 7f03a9ec8700 1 -- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230] conn(0x7f038c0aaa10 msgr2=0x7f038c0ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.619+0000 7f03a9ec8700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230] conn(0x7f038c0aaa10 0x7f038c0ace30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.619+0000 7f03a9ec8700 1 -- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f038c0a1230 msgr2=0x7f038c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.619+0000 7f03a9ec8700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f038c0a1230 0x7f038c0a36f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f039400b5c0 tx=0x7f0394005960 comp rx=0 tx=0).stop 2024-05-08T22:35:33.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.619+0000 7f03a9ec8700 1 -- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a40692f0 msgr2=0x7f03a41004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.619+0000 7f03a9ec8700 1 --2- 172.21.15.62:0/3886414344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a40692f0 0x7f03a41004e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f039800d950 tx=0x7f039800dc60 comp rx=0 tx=0).stop 2024-05-08T22:35:33.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.619+0000 7f03a3fff700 1 -- 172.21.15.62:0/3886414344 reap_dead start 2024-05-08T22:35:33.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.619+0000 7f03a9ec8700 1 -- 172.21.15.62:0/3886414344 shutdown_connections 2024-05-08T22:35:33.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.619+0000 7f03a9ec8700 1 -- 172.21.15.62:0/3886414344 >> 172.21.15.62:0/3886414344 conn(0x7f03a4075f80 msgr2=0x7f03a40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.619+0000 7f03a9ec8700 1 -- 172.21.15.62:0/3886414344 shutdown_connections 2024-05-08T22:35:33.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.619+0000 7f03a9ec8700 1 -- 172.21.15.62:0/3886414344 wait complete. 2024-05-08T22:35:33.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.578+0000 7f94c0199700 1 -- 172.21.15.62:0/1000833524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8107020 msgr2=0x7f94b8107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.578+0000 7f94c0199700 1 --2- 172.21.15.62:0/1000833524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8107020 0x7f94b8107400 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009990 tx=0x7f94b4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.578+0000 7f94c0199700 1 -- 172.21.15.62:0/1000833524 shutdown_connections 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.578+0000 7f94c0199700 1 --2- 172.21.15.62:0/1000833524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b80692f0 0x7f94b810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.578+0000 7f94c0199700 1 --2- 172.21.15.62:0/1000833524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8107020 0x7f94b8107400 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.578+0000 7f94c0199700 1 --2- 172.21.15.62:0/1000833524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94b8068950 0x7f94b8068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.578+0000 7f94c0199700 1 -- 172.21.15.62:0/1000833524 >> 172.21.15.62:0/1000833524 conn(0x7f94b8075f80 msgr2=0x7f94b8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.578+0000 7f94c0199700 1 -- 172.21.15.62:0/1000833524 shutdown_connections 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.578+0000 7f94c0199700 1 -- 172.21.15.62:0/1000833524 wait complete. 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.578+0000 7f94c0199700 1 Processor -- start 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94c0199700 1 -- start start 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94c0199700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94b8068950 0x7f94b81973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94c0199700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b80692f0 0x7f94b81978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94c0199700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8107020 0x7f94b819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94c0199700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b819c2b0 con 0x7f94b8107020 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94c0199700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b819c420 con 0x7f94b8068950 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94c0199700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b819c5e0 con 0x7f94b80692f0 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94be736700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8107020 0x7f94b819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94be736700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8107020 0x7f94b819bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57298/0 (socket says 172.21.15.62:57298) 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94be736700 1 -- 172.21.15.62:0/3982293420 learned_addr learned my addr 172.21.15.62:0/3982293420 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94bd734700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b80692f0 0x7f94b81978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94bdf35700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94b8068950 0x7f94b81973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94be736700 1 -- 172.21.15.62:0/3982293420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b80692f0 msgr2=0x7f94b81978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94be736700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b80692f0 0x7f94b81978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94be736700 1 -- 172.21.15.62:0/3982293420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94b8068950 msgr2=0x7f94b81973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94be736700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94b8068950 0x7f94b81973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94be736700 1 -- 172.21.15.62:0/3982293420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b40095f0 con 0x7f94b8107020 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.579+0000 7f94be736700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8107020 0x7f94b819bca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f94b0007ee0 tx=0x7f94b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.580+0000 7f94aeffd700 1 -- 172.21.15.62:0/3982293420 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b000dd20 con 0x7f94b8107020 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.580+0000 7f94c0199700 1 -- 172.21.15.62:0/3982293420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b819c8c0 con 0x7f94b8107020 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.580+0000 7f94aeffd700 1 -- 172.21.15.62:0/3982293420 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94b000f040 con 0x7f94b8107020 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.580+0000 7f94aeffd700 1 -- 172.21.15.62:0/3982293420 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00048a0 con 0x7f94b8107020 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.580+0000 7f94c0199700 1 -- 172.21.15.62:0/3982293420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b81a4120 con 0x7f94b8107020 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.581+0000 7f94c0199700 1 -- 172.21.15.62:0/3982293420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94b81026d0 con 0x7f94b8107020 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.582+0000 7f94aeffd700 1 -- 172.21.15.62:0/3982293420 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f94b0003a00 con 0x7f94b8107020 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.582+0000 7f94aeffd700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f94a40a11b0 0x7f94a40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.583+0000 7f94bdf35700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f94a40a11b0 0x7f94a40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.583+0000 7f94aeffd700 1 -- 172.21.15.62:0/3982293420 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f94b00ccba0 con 0x7f94b8107020 2024-05-08T22:35:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.583+0000 7f94aeffd700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078] conn(0x7f94a40aa990 0x7f94a40acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.583+0000 7f94aeffd700 1 -- 172.21.15.62:0/3982293420 --> [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94a40ad460 con 0x7f94a40aa990 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.583+0000 7f94aeffd700 1 -- 172.21.15.62:0/3982293420 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94b00cd040 con 0x7f94b8107020 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.583+0000 7f94bdf35700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f94a40a11b0 0x7f94a40a3670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94b400b5c0 tx=0x7f94b4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.583+0000 7f94bd734700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078] conn(0x7f94a40aa990 0x7f94a40acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.584+0000 7f94bd734700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078] conn(0x7f94a40aa990 0x7f94a40acdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.584+0000 7f94aeffd700 1 -- 172.21.15.62:0/3982293420 <== osd.11 v2:172.21.15.79:6824/1561471078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f94a40ad460 con 0x7f94a40aa990 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.602+0000 7f94c0199700 1 -- 172.21.15.62:0/3982293420 --> [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f94b8111160 con 0x7f94a40aa990 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94aeffd700 1 -- 172.21.15.62:0/3982293420 <== osd.11 v2:172.21.15.79:6824/1561471078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f94b8111160 con 0x7f94a40aa990 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94c0199700 1 -- 172.21.15.62:0/3982293420 >> [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078] conn(0x7f94a40aa990 msgr2=0x7f94a40acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94c0199700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078] conn(0x7f94a40aa990 0x7f94a40acdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94c0199700 1 -- 172.21.15.62:0/3982293420 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f94a40a11b0 msgr2=0x7f94a40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94c0199700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f94a40a11b0 0x7f94a40a3670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94b400b5c0 tx=0x7f94b4005960 comp rx=0 tx=0).stop 2024-05-08T22:35:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94c0199700 1 -- 172.21.15.62:0/3982293420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8107020 msgr2=0x7f94b819bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94c0199700 1 --2- 172.21.15.62:0/3982293420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8107020 0x7f94b819bca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f94b0007ee0 tx=0x7f94b000d3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94be736700 1 -- 172.21.15.62:0/3982293420 reap_dead start 2024-05-08T22:35:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94c0199700 1 -- 172.21.15.62:0/3982293420 shutdown_connections 2024-05-08T22:35:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94c0199700 1 -- 172.21.15.62:0/3982293420 >> 172.21.15.62:0/3982293420 conn(0x7f94b8075f80 msgr2=0x7f94b80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94c0199700 1 -- 172.21.15.62:0/3982293420 shutdown_connections 2024-05-08T22:35:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.603+0000 7f94c0199700 1 -- 172.21.15.62:0/3982293420 wait complete. 2024-05-08T22:35:33.813 INFO:teuthology.orchestra.run.smithi062.stdout:77309411374 2024-05-08T22:35:33.813 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-08T22:35:34.073 INFO:teuthology.orchestra.run.smithi062.stdout:133143986215 2024-05-08T22:35:34.074 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-08T22:35:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:33 smithi079 ceph-mon[46738]: pgmap v233: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-08T22:35:34.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.982+0000 7f86bdf4b700 1 -- 172.21.15.62:0/1995862298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b80691e0 msgr2=0x7f86b80695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.982+0000 7f86bdf4b700 1 --2- 172.21.15.62:0/1995862298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b80691e0 0x7f86b80695c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009990 tx=0x7f86a8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.982+0000 7f86bdf4b700 1 -- 172.21.15.62:0/1995862298 shutdown_connections 2024-05-08T22:35:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.982+0000 7f86bdf4b700 1 --2- 172.21.15.62:0/1995862298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b810da40 0x7f86b810fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.982+0000 7f86bdf4b700 1 --2- 172.21.15.62:0/1995862298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b80691e0 0x7f86b80695c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.982+0000 7f86bdf4b700 1 --2- 172.21.15.62:0/1995862298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86b8069b00 0x7f86b810d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.982+0000 7f86bdf4b700 1 -- 172.21.15.62:0/1995862298 >> 172.21.15.62:0/1995862298 conn(0x7f86b8076e70 msgr2=0x7f86b8077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.982+0000 7f86bdf4b700 1 -- 172.21.15.62:0/1995862298 shutdown_connections 2024-05-08T22:35:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.982+0000 7f86bdf4b700 1 -- 172.21.15.62:0/1995862298 wait complete. 2024-05-08T22:35:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.983+0000 7f86bdf4b700 1 Processor -- start 2024-05-08T22:35:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.983+0000 7f86bdf4b700 1 -- start start 2024-05-08T22:35:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86bdf4b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b80691e0 0x7f86b81995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86bdf4b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b8069b00 0x7f86b8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86bdf4b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86b810da40 0x7f86b819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86bdf4b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b819e4a0 con 0x7f86b80691e0 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86bdf4b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b819e610 con 0x7f86b8069b00 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86bdf4b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b819e7d0 con 0x7f86b810da40 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b80691e0 0x7f86b81995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b80691e0 0x7f86b81995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57276/0 (socket says 172.21.15.62:57276) 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b77fe700 1 -- 172.21.15.62:0/3109496558 learned_addr learned my addr 172.21.15.62:0/3109496558 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b6ffd700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b8069b00 0x7f86b8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b7fff700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86b810da40 0x7f86b819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b77fe700 1 -- 172.21.15.62:0/3109496558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86b810da40 msgr2=0x7f86b819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b77fe700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86b810da40 0x7f86b819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b77fe700 1 -- 172.21.15.62:0/3109496558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b8069b00 msgr2=0x7f86b8199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b77fe700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86b8069b00 0x7f86b8199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b77fe700 1 -- 172.21.15.62:0/3109496558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a80095f0 con 0x7f86b80691e0 2024-05-08T22:35:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b7fff700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86b810da40 0x7f86b819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b77fe700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b80691e0 0x7f86b81995a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f86a80098e0 tx=0x7f86a800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.984+0000 7f86b4ff9700 1 -- 172.21.15.62:0/3109496558 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a801d070 con 0x7f86b80691e0 2024-05-08T22:35:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.985+0000 7f86bdf4b700 1 -- 172.21.15.62:0/3109496558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b819ea50 con 0x7f86b80691e0 2024-05-08T22:35:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.985+0000 7f86b4ff9700 1 -- 172.21.15.62:0/3109496558 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86a80050d0 con 0x7f86b80691e0 2024-05-08T22:35:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.985+0000 7f86b4ff9700 1 -- 172.21.15.62:0/3109496558 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8003d20 con 0x7f86b80691e0 2024-05-08T22:35:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.985+0000 7f86bdf4b700 1 -- 172.21.15.62:0/3109496558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b81a6310 con 0x7f86b80691e0 2024-05-08T22:35:34.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.985+0000 7f86bdf4b700 1 -- 172.21.15.62:0/3109496558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f86b8113390 con 0x7f86b80691e0 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.988+0000 7f86b4ff9700 1 -- 172.21.15.62:0/3109496558 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f86a8005240 con 0x7f86b80691e0 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.988+0000 7f86b4ff9700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86a00a1230 0x7f86a00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.988+0000 7f86b4ff9700 1 -- 172.21.15.62:0/3109496558 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f86a80cdc90 con 0x7f86b80691e0 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.988+0000 7f86b6ffd700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86a00a1230 0x7f86a00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.989+0000 7f86b4ff9700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211] conn(0x7f86a00aaa10 0x7f86a00ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.989+0000 7f86b4ff9700 1 -- 172.21.15.62:0/3109496558 --> [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86a00ad4e0 con 0x7f86a00aaa10 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.989+0000 7f86b4ff9700 1 -- 172.21.15.62:0/3109496558 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86a80ce040 con 0x7f86b80691e0 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.989+0000 7f86b6ffd700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86a00a1230 0x7f86a00a36f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f86b819ab00 tx=0x7f86ac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.989+0000 7f86b7fff700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211] conn(0x7f86a00aaa10 0x7f86a00ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.990+0000 7f86b7fff700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211] conn(0x7f86a00aaa10 0x7f86a00ace30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:32.991+0000 7f86b4ff9700 1 -- 172.21.15.62:0/3109496558 <== osd.5 v2:172.21.15.79:6808/1027194211 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f86a00ad4e0 con 0x7f86a00aaa10 2024-05-08T22:35:34.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.017+0000 7f86bdf4b700 1 -- 172.21.15.62:0/3109496558 --> [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f86b819ed20 con 0x7f86a00aaa10 2024-05-08T22:35:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.018+0000 7f86b4ff9700 1 -- 172.21.15.62:0/3109496558 <== osd.5 v2:172.21.15.79:6808/1027194211 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f86b00097e0 con 0x7f86a00aaa10 2024-05-08T22:35:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.018+0000 7f86bdf4b700 1 -- 172.21.15.62:0/3109496558 >> [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211] conn(0x7f86a00aaa10 msgr2=0x7f86a00ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.018+0000 7f86bdf4b700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211] conn(0x7f86a00aaa10 0x7f86a00ace30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.019+0000 7f86bdf4b700 1 -- 172.21.15.62:0/3109496558 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86a00a1230 msgr2=0x7f86a00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.019+0000 7f86bdf4b700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86a00a1230 0x7f86a00a36f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f86b819ab00 tx=0x7f86ac00b410 comp rx=0 tx=0).stop 2024-05-08T22:35:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.019+0000 7f86bdf4b700 1 -- 172.21.15.62:0/3109496558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b80691e0 msgr2=0x7f86b81995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.019+0000 7f86bdf4b700 1 --2- 172.21.15.62:0/3109496558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b80691e0 0x7f86b81995a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f86a80098e0 tx=0x7f86a800f690 comp rx=0 tx=0).stop 2024-05-08T22:35:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.019+0000 7f86b7fff700 1 -- 172.21.15.62:0/3109496558 reap_dead start 2024-05-08T22:35:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.019+0000 7f86bdf4b700 1 -- 172.21.15.62:0/3109496558 shutdown_connections 2024-05-08T22:35:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.019+0000 7f86bdf4b700 1 -- 172.21.15.62:0/3109496558 >> 172.21.15.62:0/3109496558 conn(0x7f86b8076e70 msgr2=0x7f86b810f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.019+0000 7f86bdf4b700 1 -- 172.21.15.62:0/3109496558 shutdown_connections 2024-05-08T22:35:34.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:33.019+0000 7f86bdf4b700 1 -- 172.21.15.62:0/3109496558 wait complete. 2024-05-08T22:35:34.173 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:33 smithi062 ceph-mon[42571]: pgmap v233: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-08T22:35:34.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:33 smithi066 ceph-mon[46830]: pgmap v233: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-08T22:35:34.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.819+0000 7fd14abed700 1 -- 172.21.15.62:0/2925567243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd14410d9f0 msgr2=0x7fd14410fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.819+0000 7fd14abed700 1 --2- 172.21.15.62:0/2925567243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd14410d9f0 0x7fd14410fde0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00b0a0 tx=0x7fd13c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.819+0000 7fd14abed700 1 -- 172.21.15.62:0/2925567243 shutdown_connections 2024-05-08T22:35:34.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.819+0000 7fd14abed700 1 --2- 172.21.15.62:0/2925567243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd14410d9f0 0x7fd14410fde0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.819+0000 7fd14abed700 1 --2- 172.21.15.62:0/2925567243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd144069680 0x7fd144069a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.819+0000 7fd14abed700 1 --2- 172.21.15.62:0/2925567243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd144069fa0 0x7fd14410d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.819+0000 7fd14abed700 1 -- 172.21.15.62:0/2925567243 >> 172.21.15.62:0/2925567243 conn(0x7fd144076ec0 msgr2=0x7fd1440772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:34.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.819+0000 7fd14abed700 1 -- 172.21.15.62:0/2925567243 shutdown_connections 2024-05-08T22:35:34.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.819+0000 7fd14abed700 1 -- 172.21.15.62:0/2925567243 wait complete. 2024-05-08T22:35:34.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd14abed700 1 Processor -- start 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd14abed700 1 -- start start 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd14abed700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd144069680 0x7fd1441996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd14abed700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd144069fa0 0x7fd144199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd14abed700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd14410d9f0 0x7fd14419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd14abed700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14419e570 con 0x7fd144069680 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd14abed700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14419e6e0 con 0x7fd144069fa0 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd14abed700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14419e8a0 con 0x7fd14410d9f0 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd148989700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd144069680 0x7fd1441996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd148989700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd144069680 0x7fd1441996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57344/0 (socket says 172.21.15.62:57344) 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd148989700 1 -- 172.21.15.62:0/2063759304 learned_addr learned my addr 172.21.15.62:0/2063759304 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd14918a700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd14410d9f0 0x7fd14419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd143fff700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd144069fa0 0x7fd144199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd148989700 1 -- 172.21.15.62:0/2063759304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd14410d9f0 msgr2=0x7fd14419df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd148989700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd14410d9f0 0x7fd14419df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd148989700 1 -- 172.21.15.62:0/2063759304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd144069fa0 msgr2=0x7fd144199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd148989700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd144069fa0 0x7fd144199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd148989700 1 -- 172.21.15.62:0/2063759304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c009d00 con 0x7fd144069680 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd14918a700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd14410d9f0 0x7fd14419df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd143fff700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd144069fa0 0x7fd144199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.820+0000 7fd148989700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd144069680 0x7fd1441996c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd13800e9a0 tx=0x7fd13800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.821+0000 7fd141ffb700 1 -- 172.21.15.62:0/2063759304 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138009790 con 0x7fd144069680 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.821+0000 7fd141ffb700 1 -- 172.21.15.62:0/2063759304 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd13800cbd0 con 0x7fd144069680 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.821+0000 7fd14abed700 1 -- 172.21.15.62:0/2063759304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14419eb80 con 0x7fd144069680 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.821+0000 7fd141ffb700 1 -- 172.21.15.62:0/2063759304 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138018680 con 0x7fd144069680 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.821+0000 7fd14abed700 1 -- 172.21.15.62:0/2063759304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1441a63e0 con 0x7fd144069680 2024-05-08T22:35:34.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.822+0000 7fd14abed700 1 -- 172.21.15.62:0/2063759304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd144113330 con 0x7fd144069680 2024-05-08T22:35:34.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.823+0000 7fd141ffb700 1 -- 172.21.15.62:0/2063759304 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd13800cd40 con 0x7fd144069680 2024-05-08T22:35:34.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.823+0000 7fd141ffb700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd1340a1230 0x7fd1340a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.824+0000 7fd143fff700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd1340a1230 0x7fd1340a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.824+0000 7fd141ffb700 1 -- 172.21.15.62:0/2063759304 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd138014070 con 0x7fd144069680 2024-05-08T22:35:34.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.824+0000 7fd141ffb700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054] conn(0x7fd1340aaa10 0x7fd1340ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.824+0000 7fd141ffb700 1 -- 172.21.15.62:0/2063759304 --> [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1340ad4e0 con 0x7fd1340aaa10 2024-05-08T22:35:34.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.824+0000 7fd141ffb700 1 -- 172.21.15.62:0/2063759304 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1380cd690 con 0x7fd144069680 2024-05-08T22:35:34.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.824+0000 7fd143fff700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd1340a1230 0x7fd1340a36f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd130005fd0 tx=0x7fd130005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:34.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.824+0000 7fd14918a700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054] conn(0x7fd1340aaa10 0x7fd1340ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.824+0000 7fd14918a700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054] conn(0x7fd1340aaa10 0x7fd1340ace30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:34.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.825+0000 7fd141ffb700 1 -- 172.21.15.62:0/2063759304 <== osd.1 v2:172.21.15.66:6800/1982485054 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd1340ad4e0 con 0x7fd1340aaa10 2024-05-08T22:35:34.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.841+0000 7fd14abed700 1 -- 172.21.15.62:0/2063759304 --> [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd144100490 con 0x7fd1340aaa10 2024-05-08T22:35:34.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd141ffb700 1 -- 172.21.15.62:0/2063759304 <== osd.1 v2:172.21.15.66:6800/1982485054 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd144100490 con 0x7fd1340aaa10 2024-05-08T22:35:34.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd14abed700 1 -- 172.21.15.62:0/2063759304 >> [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054] conn(0x7fd1340aaa10 msgr2=0x7fd1340ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd14abed700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054] conn(0x7fd1340aaa10 0x7fd1340ace30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd14abed700 1 -- 172.21.15.62:0/2063759304 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd1340a1230 msgr2=0x7fd1340a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd14abed700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd1340a1230 0x7fd1340a36f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd130005fd0 tx=0x7fd130005d40 comp rx=0 tx=0).stop 2024-05-08T22:35:34.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd14abed700 1 -- 172.21.15.62:0/2063759304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd144069680 msgr2=0x7fd1441996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd14abed700 1 --2- 172.21.15.62:0/2063759304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd144069680 0x7fd1441996c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd13800e9a0 tx=0x7fd13800ed60 comp rx=0 tx=0).stop 2024-05-08T22:35:34.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd14918a700 1 -- 172.21.15.62:0/2063759304 reap_dead start 2024-05-08T22:35:34.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd14abed700 1 -- 172.21.15.62:0/2063759304 shutdown_connections 2024-05-08T22:35:34.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd14abed700 1 -- 172.21.15.62:0/2063759304 >> 172.21.15.62:0/2063759304 conn(0x7fd144076ec0 msgr2=0x7fd14410ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:34.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd14abed700 1 -- 172.21.15.62:0/2063759304 shutdown_connections 2024-05-08T22:35:34.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.842+0000 7fd14abed700 1 -- 172.21.15.62:0/2063759304 wait complete. 2024-05-08T22:35:34.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.937+0000 7fe554ba9700 1 -- 172.21.15.62:0/3557005754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe550068950 msgr2=0x7fe550068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.937+0000 7fe554ba9700 1 --2- 172.21.15.62:0/3557005754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe550068950 0x7fe550068db0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe544009990 tx=0x7fe544009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.937+0000 7fe554ba9700 1 -- 172.21.15.62:0/3557005754 shutdown_connections 2024-05-08T22:35:34.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.937+0000 7fe554ba9700 1 --2- 172.21.15.62:0/3557005754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5500692f0 0x7fe55010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.937+0000 7fe554ba9700 1 --2- 172.21.15.62:0/3557005754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe550107020 0x7fe550107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.937+0000 7fe554ba9700 1 --2- 172.21.15.62:0/3557005754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe550068950 0x7fe550068db0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.937+0000 7fe554ba9700 1 -- 172.21.15.62:0/3557005754 >> 172.21.15.62:0/3557005754 conn(0x7fe550075f80 msgr2=0x7fe550076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:34.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.937+0000 7fe554ba9700 1 -- 172.21.15.62:0/3557005754 shutdown_connections 2024-05-08T22:35:34.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.937+0000 7fe554ba9700 1 -- 172.21.15.62:0/3557005754 wait complete. 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe554ba9700 1 Processor -- start 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe554ba9700 1 -- start start 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe554ba9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe550068950 0x7fe5501973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe554ba9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5500692f0 0x7fe550197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe554ba9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe550107020 0x7fe55019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe554ba9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55019c2a0 con 0x7fe550107020 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe554ba9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55019c410 con 0x7fe5500692f0 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe554ba9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55019c5d0 con 0x7fe550068950 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe54ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe550107020 0x7fe55019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe54ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe550107020 0x7fe55019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57364/0 (socket says 172.21.15.62:57364) 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe54ed9d700 1 -- 172.21.15.62:0/3197814803 learned_addr learned my addr 172.21.15.62:0/3197814803 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.938+0000 7fe54e59c700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe550068950 0x7fe5501973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe54dd9b700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5500692f0 0x7fe550197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe54ed9d700 1 -- 172.21.15.62:0/3197814803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe550068950 msgr2=0x7fe5501973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe54ed9d700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe550068950 0x7fe5501973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe54ed9d700 1 -- 172.21.15.62:0/3197814803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5500692f0 msgr2=0x7fe550197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe54ed9d700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5500692f0 0x7fe550197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe54ed9d700 1 -- 172.21.15.62:0/3197814803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5440095f0 con 0x7fe550107020 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe54dd9b700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5500692f0 0x7fe550197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe54ed9d700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe550107020 0x7fe55019bc90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe548007ee0 tx=0x7fe54800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe53f7fe700 1 -- 172.21.15.62:0/3197814803 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548017070 con 0x7fe550107020 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe554ba9700 1 -- 172.21.15.62:0/3197814803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55019c8b0 con 0x7fe550107020 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe53f7fe700 1 -- 172.21.15.62:0/3197814803 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe54800f040 con 0x7fe550107020 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe53f7fe700 1 -- 172.21.15.62:0/3197814803 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548005740 con 0x7fe550107020 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.939+0000 7fe554ba9700 1 -- 172.21.15.62:0/3197814803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5501000a0 con 0x7fe550107020 2024-05-08T22:35:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.940+0000 7fe554ba9700 1 -- 172.21.15.62:0/3197814803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe5501111a0 con 0x7fe550107020 2024-05-08T22:35:34.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.941+0000 7fe53f7fe700 1 -- 172.21.15.62:0/3197814803 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe54800dc60 con 0x7fe550107020 2024-05-08T22:35:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.942+0000 7fe53f7fe700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe5380a1230 0x7fe5380a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.942+0000 7fe54e59c700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe5380a1230 0x7fe5380a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.942+0000 7fe53f7fe700 1 -- 172.21.15.62:0/3197814803 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe5480cc6e0 con 0x7fe550107020 2024-05-08T22:35:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.942+0000 7fe53f7fe700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351] conn(0x7fe5380aaa10 0x7fe5380ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.942+0000 7fe53f7fe700 1 -- 172.21.15.62:0/3197814803 --> [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5380ad4e0 con 0x7fe5380aaa10 2024-05-08T22:35:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.942+0000 7fe53f7fe700 1 -- 172.21.15.62:0/3197814803 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5480ccb80 con 0x7fe550107020 2024-05-08T22:35:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.942+0000 7fe54dd9b700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351] conn(0x7fe5380aaa10 0x7fe5380ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.942+0000 7fe54e59c700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe5380a1230 0x7fe5380a36f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe540009be0 tx=0x7fe540009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.943+0000 7fe54dd9b700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351] conn(0x7fe5380aaa10 0x7fe5380ace30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.943+0000 7fe53f7fe700 1 -- 172.21.15.62:0/3197814803 <== osd.9 v2:172.21.15.62:6826/1806941351 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe5380ad4e0 con 0x7fe5380aaa10 2024-05-08T22:35:34.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.959+0000 7fe554ba9700 1 -- 172.21.15.62:0/3197814803 --> [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe5501026d0 con 0x7fe5380aaa10 2024-05-08T22:35:34.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.959+0000 7fe53f7fe700 1 -- 172.21.15.62:0/3197814803 <== osd.9 v2:172.21.15.62:6826/1806941351 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe5501026d0 con 0x7fe5380aaa10 2024-05-08T22:35:34.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.960+0000 7fe554ba9700 1 -- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351] conn(0x7fe5380aaa10 msgr2=0x7fe5380ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.960+0000 7fe554ba9700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351] conn(0x7fe5380aaa10 0x7fe5380ace30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.960+0000 7fe554ba9700 1 -- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe5380a1230 msgr2=0x7fe5380a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.960+0000 7fe554ba9700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe5380a1230 0x7fe5380a36f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe540009be0 tx=0x7fe540009260 comp rx=0 tx=0).stop 2024-05-08T22:35:34.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.960+0000 7fe554ba9700 1 -- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe550107020 msgr2=0x7fe55019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:34.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.960+0000 7fe554ba9700 1 --2- 172.21.15.62:0/3197814803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe550107020 0x7fe55019bc90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe548007ee0 tx=0x7fe54800d3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:34.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.960+0000 7fe54ed9d700 1 -- 172.21.15.62:0/3197814803 reap_dead start 2024-05-08T22:35:34.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.960+0000 7fe554ba9700 1 -- 172.21.15.62:0/3197814803 shutdown_connections 2024-05-08T22:35:34.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.960+0000 7fe554ba9700 1 -- 172.21.15.62:0/3197814803 >> 172.21.15.62:0/3197814803 conn(0x7fe550075f80 msgr2=0x7fe5500fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:34.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.960+0000 7fe554ba9700 1 -- 172.21.15.62:0/3197814803 shutdown_connections 2024-05-08T22:35:34.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:34.960+0000 7fe554ba9700 1 -- 172.21.15.62:0/3197814803 wait complete. 2024-05-08T22:35:35.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.137+0000 7faa67280700 1 -- 172.21.15.62:0/1828538638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa60107020 msgr2=0x7faa60107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.137+0000 7faa67280700 1 --2- 172.21.15.62:0/1828538638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa60107020 0x7faa60107400 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b0a0 tx=0x7faa5000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.137+0000 7faa67280700 1 -- 172.21.15.62:0/1828538638 shutdown_connections 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.137+0000 7faa67280700 1 --2- 172.21.15.62:0/1828538638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa600692f0 0x7faa6010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.137+0000 7faa67280700 1 --2- 172.21.15.62:0/1828538638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa60107020 0x7faa60107400 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.137+0000 7faa67280700 1 --2- 172.21.15.62:0/1828538638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa60068950 0x7faa60068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.137+0000 7faa67280700 1 -- 172.21.15.62:0/1828538638 >> 172.21.15.62:0/1828538638 conn(0x7faa60075f80 msgr2=0x7faa60076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.137+0000 7faa67280700 1 -- 172.21.15.62:0/1828538638 shutdown_connections 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.137+0000 7faa67280700 1 -- 172.21.15.62:0/1828538638 wait complete. 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa67280700 1 Processor -- start 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa67280700 1 -- start start 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa67280700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa60068950 0x7faa60197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa67280700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa600692f0 0x7faa60197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa67280700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa60107020 0x7faa6019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa67280700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6019c410 con 0x7faa60068950 2024-05-08T22:35:35.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa67280700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6019c580 con 0x7faa600692f0 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa67280700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6019c740 con 0x7faa60107020 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa6501c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa60068950 0x7faa60197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa6501c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa60068950 0x7faa60197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57392/0 (socket says 172.21.15.62:57392) 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa6501c700 1 -- 172.21.15.62:0/4057000436 learned_addr learned my addr 172.21.15.62:0/4057000436 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.138+0000 7faa6481b700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa600692f0 0x7faa60197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa6581d700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa60107020 0x7faa6019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa6501c700 1 -- 172.21.15.62:0/4057000436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa60107020 msgr2=0x7faa6019be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa6501c700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa60107020 0x7faa6019be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa6501c700 1 -- 172.21.15.62:0/4057000436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa600692f0 msgr2=0x7faa60197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa6501c700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa600692f0 0x7faa60197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa6501c700 1 -- 172.21.15.62:0/4057000436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa50009d00 con 0x7faa60068950 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa6481b700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa600692f0 0x7faa60197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa6581d700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa60107020 0x7faa6019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:35.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa6501c700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa60068950 0x7faa60197510 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b780 tx=0x7faa500079f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa5e7fc700 1 -- 172.21.15.62:0/4057000436 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5000e040 con 0x7faa60068950 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa5e7fc700 1 -- 172.21.15.62:0/4057000436 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa50004c50 con 0x7faa60068950 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa67280700 1 -- 172.21.15.62:0/4057000436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6019c9c0 con 0x7faa60068950 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa5e7fc700 1 -- 172.21.15.62:0/4057000436 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa500052a0 con 0x7faa60068950 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.139+0000 7faa67280700 1 -- 172.21.15.62:0/4057000436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa601a4280 con 0x7faa60068950 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.140+0000 7faa67280700 1 -- 172.21.15.62:0/4057000436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa60111160 con 0x7faa60068950 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.141+0000 7faa5e7fc700 1 -- 172.21.15.62:0/4057000436 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faa5001d470 con 0x7faa60068950 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.141+0000 7faa5e7fc700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faa480a11b0 0x7faa480a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.142+0000 7faa6481b700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faa480a11b0 0x7faa480a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.142+0000 7faa5e7fc700 1 -- 172.21.15.62:0/4057000436 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faa500ce320 con 0x7faa60068950 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.142+0000 7faa5e7fc700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645] conn(0x7faa480aa990 0x7faa480acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.142+0000 7faa5e7fc700 1 -- 172.21.15.62:0/4057000436 --> [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa480ad460 con 0x7faa480aa990 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.142+0000 7faa5e7fc700 1 -- 172.21.15.62:0/4057000436 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa500ce7c0 con 0x7faa60068950 2024-05-08T22:35:35.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.142+0000 7faa6581d700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645] conn(0x7faa480aa990 0x7faa480acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.142+0000 7faa6481b700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faa480a11b0 0x7faa480a3670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faa60198a70 tx=0x7faa54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:35.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.142+0000 7faa6581d700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645] conn(0x7faa480aa990 0x7faa480acdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:35.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.143+0000 7faa5e7fc700 1 -- 172.21.15.62:0/4057000436 <== osd.0 v2:172.21.15.62:6802/2221009645 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7faa480ad460 con 0x7faa480aa990 2024-05-08T22:35:35.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.159+0000 7faa67280700 1 -- 172.21.15.62:0/4057000436 --> [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faa601026d0 con 0x7faa480aa990 2024-05-08T22:35:35.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa5e7fc700 1 -- 172.21.15.62:0/4057000436 <== osd.0 v2:172.21.15.62:6802/2221009645 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faa601026d0 con 0x7faa480aa990 2024-05-08T22:35:35.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa67280700 1 -- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645] conn(0x7faa480aa990 msgr2=0x7faa480acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa67280700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645] conn(0x7faa480aa990 0x7faa480acdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa67280700 1 -- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faa480a11b0 msgr2=0x7faa480a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa67280700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faa480a11b0 0x7faa480a3670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faa60198a70 tx=0x7faa54009260 comp rx=0 tx=0).stop 2024-05-08T22:35:35.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa67280700 1 -- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa60068950 msgr2=0x7faa60197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa67280700 1 --2- 172.21.15.62:0/4057000436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa60068950 0x7faa60197510 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b780 tx=0x7faa500079f0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa6581d700 1 -- 172.21.15.62:0/4057000436 reap_dead start 2024-05-08T22:35:35.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa67280700 1 -- 172.21.15.62:0/4057000436 shutdown_connections 2024-05-08T22:35:35.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa67280700 1 -- 172.21.15.62:0/4057000436 >> 172.21.15.62:0/4057000436 conn(0x7faa60075f80 msgr2=0x7faa600fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:35.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa67280700 1 -- 172.21.15.62:0/4057000436 shutdown_connections 2024-05-08T22:35:35.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.160+0000 7faa67280700 1 -- 172.21.15.62:0/4057000436 wait complete. 2024-05-08T22:35:35.290 INFO:teuthology.orchestra.run.smithi062.stdout:317827579919 2024-05-08T22:35:35.291 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-08T22:35:35.495 INFO:teuthology.orchestra.run.smithi062.stdout:154618822692 2024-05-08T22:35:35.496 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-08T22:35:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.547+0000 7fbf69f03700 1 -- 172.21.15.62:0/646709345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf641019e0 msgr2=0x7fbf6410ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.547+0000 7fbf69f03700 1 --2- 172.21.15.62:0/646709345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf641019e0 0x7fbf6410ddf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00b0a0 tx=0x7fbf5c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.547+0000 7fbf69f03700 1 -- 172.21.15.62:0/646709345 shutdown_connections 2024-05-08T22:35:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.547+0000 7fbf69f03700 1 --2- 172.21.15.62:0/646709345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf641019e0 0x7fbf6410ddf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.547+0000 7fbf69f03700 1 --2- 172.21.15.62:0/646709345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf64107040 0x7fbf64107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.547+0000 7fbf69f03700 1 --2- 172.21.15.62:0/646709345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf64101020 0x7fbf641014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.547+0000 7fbf69f03700 1 -- 172.21.15.62:0/646709345 >> 172.21.15.62:0/646709345 conn(0x7fbf64078100 msgr2=0x7fbf64078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:35.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.547+0000 7fbf69f03700 1 -- 172.21.15.62:0/646709345 shutdown_connections 2024-05-08T22:35:35.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.547+0000 7fbf69f03700 1 -- 172.21.15.62:0/646709345 wait complete. 2024-05-08T22:35:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.548+0000 7fbf69f03700 1 Processor -- start 2024-05-08T22:35:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.549+0000 7fbf69f03700 1 -- start start 2024-05-08T22:35:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.549+0000 7fbf69f03700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf64101020 0x7fbf641974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.549+0000 7fbf69f03700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf641019e0 0x7fbf64197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.549+0000 7fbf69f03700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf64107040 0x7fbf6419bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.549+0000 7fbf69f03700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419c3f0 con 0x7fbf641019e0 2024-05-08T22:35:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.549+0000 7fbf69f03700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419c560 con 0x7fbf64101020 2024-05-08T22:35:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.549+0000 7fbf69f03700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419c720 con 0x7fbf64107040 2024-05-08T22:35:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.549+0000 7fbf62ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf641019e0 0x7fbf64197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.549+0000 7fbf62ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf641019e0 0x7fbf64197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57410/0 (socket says 172.21.15.62:57410) 2024-05-08T22:35:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.549+0000 7fbf62ffd700 1 -- 172.21.15.62:0/3090681408 learned_addr learned my addr 172.21.15.62:0/3090681408 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.549+0000 7fbf637fe700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf64101020 0x7fbf641974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.550+0000 7fbf62ffd700 1 -- 172.21.15.62:0/3090681408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf64107040 msgr2=0x7fbf6419bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.550+0000 7fbf62ffd700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf64107040 0x7fbf6419bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.550+0000 7fbf62ffd700 1 -- 172.21.15.62:0/3090681408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf64101020 msgr2=0x7fbf641974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.550+0000 7fbf62ffd700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf64101020 0x7fbf641974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.550+0000 7fbf62ffd700 1 -- 172.21.15.62:0/3090681408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c009d00 con 0x7fbf641019e0 2024-05-08T22:35:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.550+0000 7fbf62ffd700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf641019e0 0x7fbf64197a30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400d950 tx=0x7fbf5400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:35.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.550+0000 7fbf60ff9700 1 -- 172.21.15.62:0/3090681408 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54009940 con 0x7fbf641019e0 2024-05-08T22:35:35.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.550+0000 7fbf69f03700 1 -- 172.21.15.62:0/3090681408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6419ca00 con 0x7fbf641019e0 2024-05-08T22:35:35.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.550+0000 7fbf69f03700 1 -- 172.21.15.62:0/3090681408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf641a4260 con 0x7fbf641019e0 2024-05-08T22:35:35.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.550+0000 7fbf60ff9700 1 -- 172.21.15.62:0/3090681408 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf54010460 con 0x7fbf641019e0 2024-05-08T22:35:35.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.550+0000 7fbf60ff9700 1 -- 172.21.15.62:0/3090681408 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400f620 con 0x7fbf641019e0 2024-05-08T22:35:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.551+0000 7fbf69f03700 1 -- 172.21.15.62:0/3090681408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbf50000fc0 con 0x7fbf641019e0 2024-05-08T22:35:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.552+0000 7fbf60ff9700 1 -- 172.21.15.62:0/3090681408 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbf54009aa0 con 0x7fbf641019e0 2024-05-08T22:35:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.553+0000 7fbf60ff9700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbf4c0a11b0 0x7fbf4c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.553+0000 7fbf60ff9700 1 -- 172.21.15.62:0/3090681408 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbf540ccd00 con 0x7fbf641019e0 2024-05-08T22:35:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.553+0000 7fbf637fe700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbf4c0a11b0 0x7fbf4c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.553+0000 7fbf60ff9700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757] conn(0x7fbf4c0aa990 0x7fbf4c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.553+0000 7fbf60ff9700 1 -- 172.21.15.62:0/3090681408 --> [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf4c0ad460 con 0x7fbf4c0aa990 2024-05-08T22:35:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.553+0000 7fbf60ff9700 1 -- 172.21.15.62:0/3090681408 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf540cd1a0 con 0x7fbf641019e0 2024-05-08T22:35:35.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.553+0000 7fbf637fe700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbf4c0a11b0 0x7fbf4c0a3670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbf58005950 tx=0x7fbf5800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:35.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.554+0000 7fbf63fff700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757] conn(0x7fbf4c0aa990 0x7fbf4c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:35.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.554+0000 7fbf63fff700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757] conn(0x7fbf4c0aa990 0x7fbf4c0acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:35.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.555+0000 7fbf60ff9700 1 -- 172.21.15.62:0/3090681408 <== osd.7 v2:172.21.15.66:6816/653963757 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fbf4c0ad460 con 0x7fbf4c0aa990 2024-05-08T22:35:35.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.572+0000 7fbf69f03700 1 -- 172.21.15.62:0/3090681408 --> [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbf50002d40 con 0x7fbf4c0aa990 2024-05-08T22:35:35.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.572+0000 7fbf60ff9700 1 -- 172.21.15.62:0/3090681408 <== osd.7 v2:172.21.15.66:6816/653963757 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbf50002d40 con 0x7fbf4c0aa990 2024-05-08T22:35:35.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.572+0000 7fbf69f03700 1 -- 172.21.15.62:0/3090681408 >> [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757] conn(0x7fbf4c0aa990 msgr2=0x7fbf4c0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.572+0000 7fbf69f03700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757] conn(0x7fbf4c0aa990 0x7fbf4c0acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:35.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.573+0000 7fbf69f03700 1 -- 172.21.15.62:0/3090681408 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbf4c0a11b0 msgr2=0x7fbf4c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.573+0000 7fbf69f03700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbf4c0a11b0 0x7fbf4c0a3670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbf58005950 tx=0x7fbf5800b410 comp rx=0 tx=0).stop 2024-05-08T22:35:35.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.573+0000 7fbf69f03700 1 -- 172.21.15.62:0/3090681408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf641019e0 msgr2=0x7fbf64197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:35.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.573+0000 7fbf69f03700 1 --2- 172.21.15.62:0/3090681408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf641019e0 0x7fbf64197a30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400d950 tx=0x7fbf5400dc60 comp rx=0 tx=0).stop 2024-05-08T22:35:35.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.573+0000 7fbf63fff700 1 -- 172.21.15.62:0/3090681408 reap_dead start 2024-05-08T22:35:35.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.573+0000 7fbf69f03700 1 -- 172.21.15.62:0/3090681408 shutdown_connections 2024-05-08T22:35:35.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.573+0000 7fbf69f03700 1 -- 172.21.15.62:0/3090681408 >> 172.21.15.62:0/3090681408 conn(0x7fbf64078100 msgr2=0x7fbf640ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:35.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.573+0000 7fbf69f03700 1 -- 172.21.15.62:0/3090681408 shutdown_connections 2024-05-08T22:35:35.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:35.573+0000 7fbf69f03700 1 -- 172.21.15.62:0/3090681408 wait complete. 2024-05-08T22:35:35.748 INFO:teuthology.orchestra.run.smithi062.stdout:266287972375 2024-05-08T22:35:35.749 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-08T22:35:35.929 INFO:teuthology.orchestra.run.smithi062.stdout:184683593760 2024-05-08T22:35:35.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-08T22:35:35.937 INFO:teuthology.orchestra.run.smithi062.stdout:103079215147 2024-05-08T22:35:35.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-08T22:35:36.003 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:35 smithi062 ceph-mon[42571]: pgmap v234: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-08T22:35:36.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:35 smithi079 ceph-mon[46738]: pgmap v234: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-08T22:35:36.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:35 smithi066 ceph-mon[46830]: pgmap v234: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-08T22:35:36.325 INFO:teuthology.orchestra.run.smithi062.stdout:206158430237 2024-05-08T22:35:36.325 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-08T22:35:36.338 INFO:teuthology.orchestra.run.smithi062.stdout:34359738423 2024-05-08T22:35:36.339 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-08T22:35:36.362 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:36.464 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:37.288 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:37.761 INFO:teuthology.orchestra.run.smithi062.stdout:55834574899 2024-05-08T22:35:37.761 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-08T22:35:38.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:37 smithi062 ceph-mon[42571]: pgmap v235: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-08T22:35:38.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:37 smithi079 ceph-mon[46738]: pgmap v235: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-08T22:35:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:38.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:37 smithi066 ceph-mon[46830]: pgmap v235: 97 pgs: 97 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-08T22:35:38.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.526+0000 7f8719b98700 1 -- 172.21.15.62:0/3856889951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8714106fe0 msgr2=0x7f87141073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:38.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.526+0000 7f8719b98700 1 --2- 172.21.15.62:0/3856889951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8714106fe0 0x7f87141073c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f87040099c0 tx=0x7f8704009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.526+0000 7f8719b98700 1 -- 172.21.15.62:0/3856889951 shutdown_connections 2024-05-08T22:35:38.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.526+0000 7f8719b98700 1 --2- 172.21.15.62:0/3856889951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8714069bd0 0x7f871410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.526+0000 7f8719b98700 1 --2- 172.21.15.62:0/3856889951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8714106fe0 0x7f87141073c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.526+0000 7f8719b98700 1 --2- 172.21.15.62:0/3856889951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8714069210 0x7f8714069690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.526+0000 7f8719b98700 1 -- 172.21.15.62:0/3856889951 >> 172.21.15.62:0/3856889951 conn(0x7f8714076e70 msgr2=0x7f8714077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.526+0000 7f8719b98700 1 -- 172.21.15.62:0/3856889951 shutdown_connections 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.526+0000 7f8719b98700 1 -- 172.21.15.62:0/3856889951 wait complete. 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8719b98700 1 Processor -- start 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8719b98700 1 -- start start 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8719b98700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8714069210 0x7f87140fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8719b98700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8714069bd0 0x7f87141004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8719b98700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8714106fe0 0x7f87141009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8719b98700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87141a5730 con 0x7f8714069bd0 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8719b98700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87141a58a0 con 0x7f8714069210 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8719b98700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87141a5a60 con 0x7f8714106fe0 2024-05-08T22:35:38.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8712ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8714069bd0 0x7f87141004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:38.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8712ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8714069bd0 0x7f87141004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51516/0 (socket says 172.21.15.62:51516) 2024-05-08T22:35:38.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8712ffd700 1 -- 172.21.15.62:0/1111680142 learned_addr learned my addr 172.21.15.62:0/1111680142 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:38.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f8713fff700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8714106fe0 0x7f87141009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:38.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.527+0000 7f87137fe700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8714069210 0x7f87140fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:38.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.528+0000 7f8712ffd700 1 -- 172.21.15.62:0/1111680142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8714106fe0 msgr2=0x7f87141009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:38.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.528+0000 7f8712ffd700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8714106fe0 0x7f87141009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.528+0000 7f8712ffd700 1 -- 172.21.15.62:0/1111680142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8714069210 msgr2=0x7f87140fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:38.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.528+0000 7f8712ffd700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8714069210 0x7f87140fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.528+0000 7f8712ffd700 1 -- 172.21.15.62:0/1111680142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87040095f0 con 0x7f8714069bd0 2024-05-08T22:35:38.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.528+0000 7f8712ffd700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8714069bd0 0x7f87141004a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f870800d950 tx=0x7f870800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:38.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.528+0000 7f8710ff9700 1 -- 172.21.15.62:0/1111680142 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870800f890 con 0x7f8714069bd0 2024-05-08T22:35:38.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.528+0000 7f8710ff9700 1 -- 172.21.15.62:0/1111680142 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8708010460 con 0x7f8714069bd0 2024-05-08T22:35:38.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.528+0000 7f8710ff9700 1 -- 172.21.15.62:0/1111680142 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870800e610 con 0x7f8714069bd0 2024-05-08T22:35:38.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.528+0000 7f8719b98700 1 -- 172.21.15.62:0/1111680142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87141a5d40 con 0x7f8714069bd0 2024-05-08T22:35:38.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.528+0000 7f8719b98700 1 -- 172.21.15.62:0/1111680142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87141a61e0 con 0x7f8714069bd0 2024-05-08T22:35:38.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.529+0000 7f8719b98700 1 -- 172.21.15.62:0/1111680142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87141094e0 con 0x7f8714069bd0 2024-05-08T22:35:38.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.530+0000 7f8710ff9700 1 -- 172.21.15.62:0/1111680142 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f87080108d0 con 0x7f8714069bd0 2024-05-08T22:35:38.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.531+0000 7f8710ff9700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86fc0a1230 0x7f86fc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:38.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.531+0000 7f87137fe700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86fc0a1230 0x7f86fc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:38.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.531+0000 7f8710ff9700 1 -- 172.21.15.62:0/1111680142 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f87080cc5f0 con 0x7f8714069bd0 2024-05-08T22:35:38.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.532+0000 7f87137fe700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86fc0a1230 0x7f86fc0a36f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8704009990 tx=0x7f8704005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:38.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.536+0000 7f8710ff9700 1 -- 172.21.15.62:0/1111680142 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8708095dd0 con 0x7f8714069bd0 2024-05-08T22:35:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.576+0000 7ff9341d2700 1 -- 172.21.15.62:0/3046871559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c0692f0 msgr2=0x7ff92c10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.576+0000 7ff9341d2700 1 --2- 172.21.15.62:0/3046871559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c0692f0 0x7ff92c10de60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff92400b0a0 tx=0x7ff92400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.576+0000 7ff9341d2700 1 -- 172.21.15.62:0/3046871559 shutdown_connections 2024-05-08T22:35:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.576+0000 7ff9341d2700 1 --2- 172.21.15.62:0/3046871559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c0692f0 0x7ff92c10de60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.576+0000 7ff9341d2700 1 --2- 172.21.15.62:0/3046871559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c107020 0x7ff92c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.576+0000 7ff9341d2700 1 --2- 172.21.15.62:0/3046871559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff92c068950 0x7ff92c068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.576+0000 7ff9341d2700 1 -- 172.21.15.62:0/3046871559 >> 172.21.15.62:0/3046871559 conn(0x7ff92c075f80 msgr2=0x7ff92c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.576+0000 7ff9341d2700 1 -- 172.21.15.62:0/3046871559 shutdown_connections 2024-05-08T22:35:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.576+0000 7ff9341d2700 1 -- 172.21.15.62:0/3046871559 wait complete. 2024-05-08T22:35:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff9341d2700 1 Processor -- start 2024-05-08T22:35:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff9341d2700 1 -- start start 2024-05-08T22:35:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff9341d2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c068950 0x7ff92c0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff9341d2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c0692f0 0x7ff92c1004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff9341d2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff92c107020 0x7ff92c100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff9341d2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c1a5880 con 0x7ff92c068950 2024-05-08T22:35:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff9341d2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c1a59f0 con 0x7ff92c0692f0 2024-05-08T22:35:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff9341d2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c1a5bb0 con 0x7ff92c107020 2024-05-08T22:35:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff931f6e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c068950 0x7ff92c0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff931f6e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c068950 0x7ff92c0fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51538/0 (socket says 172.21.15.62:51538) 2024-05-08T22:35:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff931f6e700 1 -- 172.21.15.62:0/3727690034 learned_addr learned my addr 172.21.15.62:0/3727690034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff93176d700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c0692f0 0x7ff92c1004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff93276f700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff92c107020 0x7ff92c100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff931f6e700 1 -- 172.21.15.62:0/3727690034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff92c107020 msgr2=0x7ff92c100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff931f6e700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff92c107020 0x7ff92c100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff931f6e700 1 -- 172.21.15.62:0/3727690034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c0692f0 msgr2=0x7ff92c1004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff931f6e700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c0692f0 0x7ff92c1004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff931f6e700 1 -- 172.21.15.62:0/3727690034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff924009d00 con 0x7ff92c068950 2024-05-08T22:35:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.577+0000 7ff931f6e700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c068950 0x7ff92c0fffa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff92800e9a0 tx=0x7ff92800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.578+0000 7ff922ffd700 1 -- 172.21.15.62:0/3727690034 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928009790 con 0x7ff92c068950 2024-05-08T22:35:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.578+0000 7ff9341d2700 1 -- 172.21.15.62:0/3727690034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c1a5e90 con 0x7ff92c068950 2024-05-08T22:35:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.578+0000 7ff922ffd700 1 -- 172.21.15.62:0/3727690034 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff928010430 con 0x7ff92c068950 2024-05-08T22:35:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.578+0000 7ff922ffd700 1 -- 172.21.15.62:0/3727690034 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928018680 con 0x7ff92c068950 2024-05-08T22:35:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.578+0000 7ff9341d2700 1 -- 172.21.15.62:0/3727690034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c1a6350 con 0x7ff92c068950 2024-05-08T22:35:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.579+0000 7ff9341d2700 1 -- 172.21.15.62:0/3727690034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c109540 con 0x7ff92c068950 2024-05-08T22:35:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.581+0000 7ff922ffd700 1 -- 172.21.15.62:0/3727690034 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff9280105a0 con 0x7ff92c068950 2024-05-08T22:35:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.581+0000 7ff922ffd700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff9180a11b0 0x7ff9180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.581+0000 7ff93176d700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff9180a11b0 0x7ff9180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.581+0000 7ff922ffd700 1 -- 172.21.15.62:0/3727690034 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff928014070 con 0x7ff92c068950 2024-05-08T22:35:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.582+0000 7ff93176d700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff9180a11b0 0x7ff9180a3670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff92c101680 tx=0x7ff91c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:38.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.585+0000 7ff922ffd700 1 -- 172.21.15.62:0/3727690034 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff9280963c0 con 0x7ff92c068950 2024-05-08T22:35:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:38.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.761+0000 7f8719b98700 1 -- 172.21.15.62:0/1111680142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f8714102690 con 0x7f8714069bd0 2024-05-08T22:35:38.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.762+0000 7f8710ff9700 1 -- 172.21.15.62:0/1111680142 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f87080993f0 con 0x7f8714069bd0 2024-05-08T22:35:38.762 INFO:teuthology.orchestra.run.smithi062.stdout:236223201306 2024-05-08T22:35:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 -- 172.21.15.62:0/1111680142 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86fc0a1230 msgr2=0x7f86fc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86fc0a1230 0x7f86fc0a36f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8704009990 tx=0x7f8704005a90 comp rx=0 tx=0).stop 2024-05-08T22:35:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 -- 172.21.15.62:0/1111680142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8714069bd0 msgr2=0x7f87141004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8714069bd0 0x7f87141004a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f870800d950 tx=0x7f870800dc60 comp rx=0 tx=0).stop 2024-05-08T22:35:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 -- 172.21.15.62:0/1111680142 shutdown_connections 2024-05-08T22:35:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8714106fe0 0x7f87141009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86fc0a1230 0x7f86fc0a36f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8714069210 0x7f87140fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 --2- 172.21.15.62:0/1111680142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8714069bd0 0x7f87141004a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 -- 172.21.15.62:0/1111680142 >> 172.21.15.62:0/1111680142 conn(0x7f8714076e70 msgr2=0x7f87140feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 -- 172.21.15.62:0/1111680142 shutdown_connections 2024-05-08T22:35:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.764+0000 7f8719b98700 1 -- 172.21.15.62:0/1111680142 wait complete. 2024-05-08T22:35:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.819+0000 7ff9341d2700 1 -- 172.21.15.62:0/3727690034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff92c101430 con 0x7ff92c068950 2024-05-08T22:35:38.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.819+0000 7ff922ffd700 1 -- 172.21.15.62:0/3727690034 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff928026090 con 0x7ff92c068950 2024-05-08T22:35:38.819 INFO:teuthology.orchestra.run.smithi062.stdout:133143986216 2024-05-08T22:35:38.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.821+0000 7ff9341d2700 1 -- 172.21.15.62:0/3727690034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff9180a11b0 msgr2=0x7ff9180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.821+0000 7ff9341d2700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff9180a11b0 0x7ff9180a3670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff92c101680 tx=0x7ff91c0073d0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.821+0000 7ff9341d2700 1 -- 172.21.15.62:0/3727690034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c068950 msgr2=0x7ff92c0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.821+0000 7ff9341d2700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c068950 0x7ff92c0fffa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff92800e9a0 tx=0x7ff92800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.821+0000 7ff9341d2700 1 -- 172.21.15.62:0/3727690034 shutdown_connections 2024-05-08T22:35:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.821+0000 7ff9341d2700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff92c107020 0x7ff92c100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.821+0000 7ff9341d2700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c068950 0x7ff92c0fffa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.821+0000 7ff9341d2700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff9180a11b0 0x7ff9180a3670 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.821+0000 7ff9341d2700 1 --2- 172.21.15.62:0/3727690034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff92c0692f0 0x7ff92c1004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.821+0000 7ff9341d2700 1 -- 172.21.15.62:0/3727690034 >> 172.21.15.62:0/3727690034 conn(0x7ff92c075f80 msgr2=0x7ff92c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.822+0000 7ff9341d2700 1 -- 172.21.15.62:0/3727690034 shutdown_connections 2024-05-08T22:35:38.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:38.822+0000 7ff9341d2700 1 -- 172.21.15.62:0/3727690034 wait complete. 2024-05-08T22:35:39.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:38 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1111680142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T22:35:39.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:38 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3727690034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T22:35:39.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:38 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1111680142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T22:35:39.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:38 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3727690034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T22:35:39.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.056+0000 7f6b34440700 1 -- 172.21.15.62:0/1015470150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b2c0ffed0 msgr2=0x7f6b2c100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:39.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.056+0000 7f6b34440700 1 --2- 172.21.15.62:0/1015470150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b2c0ffed0 0x7f6b2c100350 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009990 tx=0x7f6b1c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.056+0000 7f6b34440700 1 -- 172.21.15.62:0/1015470150 shutdown_connections 2024-05-08T22:35:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.056+0000 7f6b34440700 1 --2- 172.21.15.62:0/1015470150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b2c100890 0x7f6b2c10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.056+0000 7f6b34440700 1 --2- 172.21.15.62:0/1015470150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b2c105f00 0x7f6b2c1062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.056+0000 7f6b34440700 1 --2- 172.21.15.62:0/1015470150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b2c0ffed0 0x7f6b2c100350 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.056+0000 7f6b34440700 1 -- 172.21.15.62:0/1015470150 >> 172.21.15.62:0/1015470150 conn(0x7f6b2c075f80 msgr2=0x7f6b2c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.056+0000 7f6b34440700 1 -- 172.21.15.62:0/1015470150 shutdown_connections 2024-05-08T22:35:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.056+0000 7f6b34440700 1 -- 172.21.15.62:0/1015470150 wait complete. 2024-05-08T22:35:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.056+0000 7f6b34440700 1 Processor -- start 2024-05-08T22:35:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b34440700 1 -- start start 2024-05-08T22:35:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b34440700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b2c0ffed0 0x7f6b2c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b34440700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b2c100890 0x7f6b2c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b34440700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b2c105f00 0x7f6b2c19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b34440700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c19e4b0 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b34440700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c19e620 con 0x7f6b2c100890 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b34440700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c19e7e0 con 0x7f6b2c105f00 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b321dc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b2c0ffed0 0x7f6b2c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b321dc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b2c0ffed0 0x7f6b2c1995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51562/0 (socket says 172.21.15.62:51562) 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b321dc700 1 -- 172.21.15.62:0/2822084495 learned_addr learned my addr 172.21.15.62:0/2822084495 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b329dd700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b2c105f00 0x7f6b2c19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b321dc700 1 -- 172.21.15.62:0/2822084495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b2c105f00 msgr2=0x7f6b2c19dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b321dc700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b2c105f00 0x7f6b2c19dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b319db700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b2c100890 0x7f6b2c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b321dc700 1 -- 172.21.15.62:0/2822084495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b2c100890 msgr2=0x7f6b2c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b321dc700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b2c100890 0x7f6b2c199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b321dc700 1 -- 172.21.15.62:0/2822084495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0095f0 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b321dc700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b2c0ffed0 0x7f6b2c1995b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800c920 tx=0x7f6b2800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b237fe700 1 -- 172.21.15.62:0/2822084495 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28007b00 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b237fe700 1 -- 172.21.15.62:0/2822084495 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b28004cf0 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.057+0000 7f6b237fe700 1 -- 172.21.15.62:0/2822084495 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28005610 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.058+0000 7f6b34440700 1 -- 172.21.15.62:0/2822084495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c19eac0 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.058+0000 7f6b34440700 1 -- 172.21.15.62:0/2822084495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c1a6320 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.058+0000 7f6b34440700 1 -- 172.21.15.62:0/2822084495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c1015b0 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.060+0000 7f6b237fe700 1 -- 172.21.15.62:0/2822084495 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6b28004750 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.060+0000 7f6b237fe700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b180a11b0 0x7f6b180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.060+0000 7f6b237fe700 1 -- 172.21.15.62:0/2822084495 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6b280cc880 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.060+0000 7f6b319db700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b180a11b0 0x7f6b180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.061+0000 7f6b319db700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b180a11b0 0x7f6b180a3670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c003730 tx=0x7f6b1c0036c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:39.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.064+0000 7f6b237fe700 1 -- 172.21.15.62:0/2822084495 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6b28096060 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.281+0000 7f6b34440700 1 -- 172.21.15.62:0/2822084495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f6b2c04ec60 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.282+0000 7f6b237fe700 1 -- 172.21.15.62:0/2822084495 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f6b2801d020 con 0x7f6b2c0ffed0 2024-05-08T22:35:39.282 INFO:teuthology.orchestra.run.smithi062.stdout:287762808852 2024-05-08T22:35:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 -- 172.21.15.62:0/2822084495 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b180a11b0 msgr2=0x7f6b180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b180a11b0 0x7f6b180a3670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c003730 tx=0x7f6b1c0036c0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 -- 172.21.15.62:0/2822084495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b2c0ffed0 msgr2=0x7f6b2c1995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b2c0ffed0 0x7f6b2c1995b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800c920 tx=0x7f6b2800cce0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 -- 172.21.15.62:0/2822084495 shutdown_connections 2024-05-08T22:35:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b2c105f00 0x7f6b2c19dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b2c0ffed0 0x7f6b2c1995b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b180a11b0 0x7f6b180a3670 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 --2- 172.21.15.62:0/2822084495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b2c100890 0x7f6b2c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 -- 172.21.15.62:0/2822084495 >> 172.21.15.62:0/2822084495 conn(0x7f6b2c075f80 msgr2=0x7f6b2c068950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:39.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 -- 172.21.15.62:0/2822084495 shutdown_connections 2024-05-08T22:35:39.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.284+0000 7f6b34440700 1 -- 172.21.15.62:0/2822084495 wait complete. 2024-05-08T22:35:39.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:38 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1111680142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T22:35:39.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:38 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3727690034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T22:35:39.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.416+0000 7f6b936fa700 1 -- 172.21.15.62:0/1137889967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b8c100890 msgr2=0x7f6b8c10cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:39.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.416+0000 7f6b936fa700 1 --2- 172.21.15.62:0/1137889967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b8c100890 0x7f6b8c10cd40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400b0a0 tx=0x7f6b8400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.416+0000 7f6b936fa700 1 -- 172.21.15.62:0/1137889967 shutdown_connections 2024-05-08T22:35:39.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.416+0000 7f6b936fa700 1 --2- 172.21.15.62:0/1137889967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b8c100890 0x7f6b8c10cd40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.416+0000 7f6b936fa700 1 --2- 172.21.15.62:0/1137889967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b8c105f00 0x7f6b8c1062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.416+0000 7f6b936fa700 1 --2- 172.21.15.62:0/1137889967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b8c0ffed0 0x7f6b8c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.416+0000 7f6b936fa700 1 -- 172.21.15.62:0/1137889967 >> 172.21.15.62:0/1137889967 conn(0x7f6b8c075f80 msgr2=0x7f6b8c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.416+0000 7f6b936fa700 1 -- 172.21.15.62:0/1137889967 shutdown_connections 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.416+0000 7f6b936fa700 1 -- 172.21.15.62:0/1137889967 wait complete. 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.416+0000 7f6b936fa700 1 Processor -- start 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.416+0000 7f6b936fa700 1 -- start start 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b936fa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b8c0ffed0 0x7f6b8c1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b936fa700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b8c100890 0x7f6b8c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b936fa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b8c105f00 0x7f6b8c19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b936fa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c19c3b0 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b936fa700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c19c520 con 0x7f6b8c100890 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b936fa700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c19c6e0 con 0x7f6b8c105f00 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b91496700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b8c0ffed0 0x7f6b8c1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b91496700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b8c0ffed0 0x7f6b8c1974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51576/0 (socket says 172.21.15.62:51576) 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b91496700 1 -- 172.21.15.62:0/1296862793 learned_addr learned my addr 172.21.15.62:0/1296862793 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b90c95700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b8c100890 0x7f6b8c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b91c97700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b8c105f00 0x7f6b8c19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b91496700 1 -- 172.21.15.62:0/1296862793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b8c105f00 msgr2=0x7f6b8c19bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b91496700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b8c105f00 0x7f6b8c19bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b91496700 1 -- 172.21.15.62:0/1296862793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b8c100890 msgr2=0x7f6b8c1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b91496700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b8c100890 0x7f6b8c1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b91496700 1 -- 172.21.15.62:0/1296862793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b84009d00 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b90c95700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b8c100890 0x7f6b8c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.417+0000 7f6b91496700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b8c0ffed0 0x7f6b8c1974b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800e9a0 tx=0x7f6b8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.418+0000 7f6b827fc700 1 -- 172.21.15.62:0/1296862793 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b88009790 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.418+0000 7f6b936fa700 1 -- 172.21.15.62:0/1296862793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8c19c9c0 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.418+0000 7f6b827fc700 1 -- 172.21.15.62:0/1296862793 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b88010430 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.418+0000 7f6b827fc700 1 -- 172.21.15.62:0/1296862793 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b88018680 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.418+0000 7f6b936fa700 1 -- 172.21.15.62:0/1296862793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8c1a4220 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.419+0000 7f6b936fa700 1 -- 172.21.15.62:0/1296862793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8c1015b0 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.420+0000 7f6b827fc700 1 -- 172.21.15.62:0/1296862793 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6b880105a0 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.420+0000 7f6b827fc700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b780a11b0 0x7f6b780a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.420+0000 7f6b90c95700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b780a11b0 0x7f6b780a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.420+0000 7f6b827fc700 1 -- 172.21.15.62:0/1296862793 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6b88014070 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.421+0000 7f6b90c95700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b780a11b0 0x7f6b780a3670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c198a10 tx=0x7f6b7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:39.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.425+0000 7f6b827fc700 1 -- 172.21.15.62:0/1296862793 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6b88052d60 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.633+0000 7f6b936fa700 1 -- 172.21.15.62:0/1296862793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6b8c10fdb0 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.634+0000 7f6b827fc700 1 -- 172.21.15.62:0/1296862793 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6b88096200 con 0x7f6b8c0ffed0 2024-05-08T22:35:39.634 INFO:teuthology.orchestra.run.smithi062.stdout:77309411375 2024-05-08T22:35:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 -- 172.21.15.62:0/1296862793 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b780a11b0 msgr2=0x7f6b780a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b780a11b0 0x7f6b780a3670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c198a10 tx=0x7f6b7c009260 comp rx=0 tx=0).stop 2024-05-08T22:35:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 -- 172.21.15.62:0/1296862793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b8c0ffed0 msgr2=0x7f6b8c1974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b8c0ffed0 0x7f6b8c1974b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800e9a0 tx=0x7f6b8800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 -- 172.21.15.62:0/1296862793 shutdown_connections 2024-05-08T22:35:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b8c105f00 0x7f6b8c19bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b8c0ffed0 0x7f6b8c1974b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6b780a11b0 0x7f6b780a3670 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 --2- 172.21.15.62:0/1296862793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b8c100890 0x7f6b8c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 -- 172.21.15.62:0/1296862793 >> 172.21.15.62:0/1296862793 conn(0x7f6b8c075f80 msgr2=0x7f6b8c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:39.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 -- 172.21.15.62:0/1296862793 shutdown_connections 2024-05-08T22:35:39.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:39.636+0000 7f6b936fa700 1 -- 172.21.15.62:0/1296862793 wait complete. 2024-05-08T22:35:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:39.915 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201305 got 236223201306 for osd.8 2024-05-08T22:35:39.915 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:40.073 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:40.074 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:40.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:39 smithi079 ceph-mon[46738]: pgmap v236: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T22:35:40.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:39 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2822084495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T22:35:40.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:39 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1296862793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T22:35:40.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:39 smithi062 ceph-mon[42571]: pgmap v236: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T22:35:40.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:39 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2822084495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T22:35:40.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:39 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1296862793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T22:35:40.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:39 smithi066 ceph-mon[46830]: pgmap v236: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T22:35:40.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:39 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2822084495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T22:35:40.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:39 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1296862793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T22:35:40.639 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411374 got 77309411375 for osd.2 2024-05-08T22:35:40.640 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:40.931 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986215 got 133143986216 for osd.4 2024-05-08T22:35:40.931 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:41.524 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808850 got 287762808852 for osd.10 2024-05-08T22:35:41.525 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:42.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.998+0000 7fcdbec0a700 1 -- 172.21.15.62:0/224702799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb81084a0 msgr2=0x7fcdb8110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.998+0000 7fcdbec0a700 1 --2- 172.21.15.62:0/224702799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb81084a0 0x7fcdb8110070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b0a0 tx=0x7fcdb000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.998+0000 7fcdbec0a700 1 -- 172.21.15.62:0/224702799 shutdown_connections 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.998+0000 7fcdbec0a700 1 --2- 172.21.15.62:0/224702799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb81084a0 0x7fcdb8110070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.998+0000 7fcdbec0a700 1 --2- 172.21.15.62:0/224702799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdb81033c0 0x7fcdb81037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.998+0000 7fcdbec0a700 1 --2- 172.21.15.62:0/224702799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb8103d70 0x7fcdb8107dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.998+0000 7fcdbec0a700 1 -- 172.21.15.62:0/224702799 >> 172.21.15.62:0/224702799 conn(0x7fcdb80fec50 msgr2=0x7fcdb8101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.998+0000 7fcdbec0a700 1 -- 172.21.15.62:0/224702799 shutdown_connections 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.998+0000 7fcdbec0a700 1 -- 172.21.15.62:0/224702799 wait complete. 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.998+0000 7fcdbec0a700 1 Processor -- start 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.998+0000 7fcdbec0a700 1 -- start start 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbec0a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb81033c0 0x7fcdb81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbec0a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb8103d70 0x7fcdb8199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbec0a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdb81084a0 0x7fcdb819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbec0a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb819e5a0 con 0x7fcdb81033c0 2024-05-08T22:35:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbec0a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb819e710 con 0x7fcdb8103d70 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbec0a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb819e8d0 con 0x7fcdb81084a0 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbc9a6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb81033c0 0x7fcdb81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbc9a6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb81033c0 0x7fcdb81996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51610/0 (socket says 172.21.15.62:51610) 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbc9a6700 1 -- 172.21.15.62:0/2176800244 learned_addr learned my addr 172.21.15.62:0/2176800244 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdb7fff700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb8103d70 0x7fcdb8199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbd1a7700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdb81084a0 0x7fcdb819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbc9a6700 1 -- 172.21.15.62:0/2176800244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdb81084a0 msgr2=0x7fcdb819df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbc9a6700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdb81084a0 0x7fcdb819df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbc9a6700 1 -- 172.21.15.62:0/2176800244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb8103d70 msgr2=0x7fcdb8199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbc9a6700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb8103d70 0x7fcdb8199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbc9a6700 1 -- 172.21.15.62:0/2176800244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0009d00 con 0x7fcdb81033c0 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdb7fff700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb8103d70 0x7fcdb8199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbc9a6700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb81033c0 0x7fcdb81996f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e9a0 tx=0x7fcdac00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdb5ffb700 1 -- 172.21.15.62:0/2176800244 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac009790 con 0x7fcdb81033c0 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdb5ffb700 1 -- 172.21.15.62:0/2176800244 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdac00cbd0 con 0x7fcdb81033c0 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdbec0a700 1 -- 172.21.15.62:0/2176800244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb819ebb0 con 0x7fcdb81033c0 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.999+0000 7fcdb5ffb700 1 -- 172.21.15.62:0/2176800244 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac018680 con 0x7fcdb81033c0 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.000+0000 7fcdbec0a700 1 -- 172.21.15.62:0/2176800244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb81a6410 con 0x7fcdb81033c0 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.001+0000 7fcdbec0a700 1 -- 172.21.15.62:0/2176800244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb810b7c0 con 0x7fcdb81033c0 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.002+0000 7fcdb5ffb700 1 -- 172.21.15.62:0/2176800244 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcdac00cd40 con 0x7fcdb81033c0 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.003+0000 7fcdb5ffb700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcda80a1230 0x7fcda80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.003+0000 7fcdb7fff700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcda80a1230 0x7fcda80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.003+0000 7fcdb5ffb700 1 -- 172.21.15.62:0/2176800244 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcdac014070 con 0x7fcdb81033c0 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.004+0000 7fcdb7fff700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcda80a1230 0x7fcda80a36f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcda4005e10 tx=0x7fcda4005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:42.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.007+0000 7fcdb5ffb700 1 -- 172.21.15.62:0/2176800244 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcdac052d60 con 0x7fcdb81033c0 2024-05-08T22:35:42.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.970+0000 7f595d89a700 1 -- 172.21.15.62:0/970581012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59580692f0 msgr2=0x7f595810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.970+0000 7f595d89a700 1 --2- 172.21.15.62:0/970581012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59580692f0 0x7f595810de60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5950009f90 tx=0x7f595000b330 comp rx=0 tx=0).stop 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.970+0000 7f595d89a700 1 -- 172.21.15.62:0/970581012 shutdown_connections 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.970+0000 7f595d89a700 1 --2- 172.21.15.62:0/970581012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59580692f0 0x7f595810de60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.970+0000 7f595d89a700 1 --2- 172.21.15.62:0/970581012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5958107020 0x7f5958107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.970+0000 7f595d89a700 1 --2- 172.21.15.62:0/970581012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5958068950 0x7f5958068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.970+0000 7f595d89a700 1 -- 172.21.15.62:0/970581012 >> 172.21.15.62:0/970581012 conn(0x7f5958075f80 msgr2=0x7f5958076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.970+0000 7f595d89a700 1 -- 172.21.15.62:0/970581012 shutdown_connections 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.970+0000 7f595d89a700 1 -- 172.21.15.62:0/970581012 wait complete. 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f595d89a700 1 Processor -- start 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f595d89a700 1 -- start start 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f595d89a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5958068950 0x7f5958197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f595d89a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59580692f0 0x7f5958197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f595d89a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5958107020 0x7f595819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f595d89a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595819c3b0 con 0x7f5958068950 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f595d89a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595819c520 con 0x7f59580692f0 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f595d89a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595819c6e0 con 0x7f5958107020 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f5956ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5958068950 0x7f5958197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f5956ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5958068950 0x7f5958197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51602/0 (socket says 172.21.15.62:51602) 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f5956ffd700 1 -- 172.21.15.62:0/3292169942 learned_addr learned my addr 172.21.15.62:0/3292169942 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f59567fc700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59580692f0 0x7f5958197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.971+0000 7f59577fe700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5958107020 0x7f595819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f5956ffd700 1 -- 172.21.15.62:0/3292169942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5958107020 msgr2=0x7f595819bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f5956ffd700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5958107020 0x7f595819bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f5956ffd700 1 -- 172.21.15.62:0/3292169942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59580692f0 msgr2=0x7f5958197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f5956ffd700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59580692f0 0x7f5958197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f5956ffd700 1 -- 172.21.15.62:0/3292169942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59480095f0 con 0x7f5958068950 2024-05-08T22:35:42.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f59567fc700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59580692f0 0x7f5958197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:42.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f5956ffd700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5958068950 0x7f5958197500 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f594800ec10 tx=0x7f594800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:42.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f595c898700 1 -- 172.21.15.62:0/3292169942 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800cc90 con 0x7f5958068950 2024-05-08T22:35:42.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f595d89a700 1 -- 172.21.15.62:0/3292169942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5950009c40 con 0x7f5958068950 2024-05-08T22:35:42.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f595c898700 1 -- 172.21.15.62:0/3292169942 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59480044e0 con 0x7f5958068950 2024-05-08T22:35:42.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f595c898700 1 -- 172.21.15.62:0/3292169942 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948010430 con 0x7f5958068950 2024-05-08T22:35:42.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.972+0000 7f595d89a700 1 -- 172.21.15.62:0/3292169942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59581a4050 con 0x7f5958068950 2024-05-08T22:35:42.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.973+0000 7f595d89a700 1 -- 172.21.15.62:0/3292169942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5958109540 con 0x7f5958068950 2024-05-08T22:35:42.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.975+0000 7f595c898700 1 -- 172.21.15.62:0/3292169942 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5948003f90 con 0x7f5958068950 2024-05-08T22:35:42.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.976+0000 7f595c898700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f59400a11b0 0x7f59400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.976+0000 7f59567fc700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f59400a11b0 0x7f59400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.976+0000 7f595c898700 1 -- 172.21.15.62:0/3292169942 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5948014070 con 0x7f5958068950 2024-05-08T22:35:42.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.977+0000 7f59567fc700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f59400a11b0 0x7f59400a3670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5958198a10 tx=0x7f594c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:42.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:41.980+0000 7f595c898700 1 -- 172.21.15.62:0/3292169942 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f59480966d0 con 0x7f5958068950 2024-05-08T22:35:42.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:41 smithi062 ceph-mon[42571]: pgmap v237: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-08T22:35:42.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.198+0000 7f595d89a700 1 -- 172.21.15.62:0/3292169942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f59581026d0 con 0x7f5958068950 2024-05-08T22:35:42.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.198+0000 7f595c898700 1 -- 172.21.15.62:0/3292169942 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f594801f340 con 0x7f5958068950 2024-05-08T22:35:42.199 INFO:teuthology.orchestra.run.smithi062.stdout:103079215148 2024-05-08T22:35:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 -- 172.21.15.62:0/3292169942 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f59400a11b0 msgr2=0x7f59400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f59400a11b0 0x7f59400a3670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5958198a10 tx=0x7f594c009260 comp rx=0 tx=0).stop 2024-05-08T22:35:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 -- 172.21.15.62:0/3292169942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5958068950 msgr2=0x7f5958197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5958068950 0x7f5958197500 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f594800ec10 tx=0x7f594800ef20 comp rx=0 tx=0).stop 2024-05-08T22:35:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 -- 172.21.15.62:0/3292169942 shutdown_connections 2024-05-08T22:35:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5958107020 0x7f595819bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5958068950 0x7f5958197500 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f59400a11b0 0x7f59400a3670 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 --2- 172.21.15.62:0/3292169942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59580692f0 0x7f5958197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 -- 172.21.15.62:0/3292169942 >> 172.21.15.62:0/3292169942 conn(0x7f5958075f80 msgr2=0x7f59580fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:42.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 -- 172.21.15.62:0/3292169942 shutdown_connections 2024-05-08T22:35:42.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.202+0000 7f595d89a700 1 -- 172.21.15.62:0/3292169942 wait complete. 2024-05-08T22:35:42.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.221+0000 7fcb80ce5700 1 -- 172.21.15.62:0/3024582667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c100a50 msgr2=0x7fcb7c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.221+0000 7fcb80ce5700 1 --2- 172.21.15.62:0/3024582667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c100a50 0x7fcb7c100ed0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb700099c0 tx=0x7fcb70009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.221+0000 7fcb80ce5700 1 -- 172.21.15.62:0/3024582667 shutdown_connections 2024-05-08T22:35:42.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.221+0000 7fcb80ce5700 1 --2- 172.21.15.62:0/3024582667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb7c101410 0x7fcb7c10cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.221+0000 7fcb80ce5700 1 --2- 172.21.15.62:0/3024582667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb7c104ed0 0x7fcb7c1052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.221+0000 7fcb80ce5700 1 --2- 172.21.15.62:0/3024582667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c100a50 0x7fcb7c100ed0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.221+0000 7fcb80ce5700 1 -- 172.21.15.62:0/3024582667 >> 172.21.15.62:0/3024582667 conn(0x7fcb7c075260 msgr2=0x7fcb7c075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:42.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.222+0000 7fcb80ce5700 1 -- 172.21.15.62:0/3024582667 shutdown_connections 2024-05-08T22:35:42.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.222+0000 7fcb80ce5700 1 -- 172.21.15.62:0/3024582667 wait complete. 2024-05-08T22:35:42.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.222+0000 7fcb80ce5700 1 Processor -- start 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.222+0000 7fcb80ce5700 1 -- start start 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb80ce5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c100a50 0x7fcb7c199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb80ce5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb7c101410 0x7fcb7c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb80ce5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb7c104ed0 0x7fcb7c19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb80ce5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c19e570 con 0x7fcb7c100a50 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb80ce5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c19e6e0 con 0x7fcb7c104ed0 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb80ce5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c19e8a0 con 0x7fcb7c101410 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb7a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c100a50 0x7fcb7c199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb7ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb7c104ed0 0x7fcb7c19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb7a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c100a50 0x7fcb7c199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51618/0 (socket says 172.21.15.62:51618) 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb7a59c700 1 -- 172.21.15.62:0/2250551423 learned_addr learned my addr 172.21.15.62:0/2250551423 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb7ad9d700 1 -- 172.21.15.62:0/2250551423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb7c101410 msgr2=0x7fcb7c199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb7ad9d700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb7c101410 0x7fcb7c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb7ad9d700 1 -- 172.21.15.62:0/2250551423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c100a50 msgr2=0x7fcb7c199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb7ad9d700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c100a50 0x7fcb7c199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb7ad9d700 1 -- 172.21.15.62:0/2250551423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700095f0 con 0x7fcb7c104ed0 2024-05-08T22:35:42.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb7a59c700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c100a50 0x7fcb7c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:42.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.223+0000 7fcb7ad9d700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb7c104ed0 0x7fcb7c19df60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400efc0 tx=0x7fcb7400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:42.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.224+0000 7fcb6b7fe700 1 -- 172.21.15.62:0/2250551423 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74018070 con 0x7fcb7c104ed0 2024-05-08T22:35:42.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.224+0000 7fcb80ce5700 1 -- 172.21.15.62:0/2250551423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7c19eb20 con 0x7fcb7c104ed0 2024-05-08T22:35:42.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.224+0000 7fcb80ce5700 1 -- 172.21.15.62:0/2250551423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7c1a63e0 con 0x7fcb7c104ed0 2024-05-08T22:35:42.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.225+0000 7fcb80ce5700 1 -- 172.21.15.62:0/2250551423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7c0684d0 con 0x7fcb7c104ed0 2024-05-08T22:35:42.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.226+0000 7fcb6b7fe700 1 -- 172.21.15.62:0/2250551423 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb7400f040 con 0x7fcb7c104ed0 2024-05-08T22:35:42.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.226+0000 7fcb6b7fe700 1 -- 172.21.15.62:0/2250551423 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400ce10 con 0x7fcb7c104ed0 2024-05-08T22:35:42.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.229+0000 7fcb6b7fe700 1 -- 172.21.15.62:0/2250551423 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcb74003a00 con 0x7fcb7c104ed0 2024-05-08T22:35:42.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.229+0000 7fcb6b7fe700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb640a11b0 0x7fcb640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.229+0000 7fcb6b7fe700 1 -- 172.21.15.62:0/2250551423 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcb740cd250 con 0x7fcb7c104ed0 2024-05-08T22:35:42.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.229+0000 7fcb7a59c700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb640a11b0 0x7fcb640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.230+0000 7fcb7a59c700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb640a11b0 0x7fcb640a3670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009ea0 tx=0x7fcb6c009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:42.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.232+0000 7fcb6b7fe700 1 -- 172.21.15.62:0/2250551423 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcb74096ac0 con 0x7fcb7c104ed0 2024-05-08T22:35:42.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.255+0000 7fcdbec0a700 1 -- 172.21.15.62:0/2176800244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fcdb81048a0 con 0x7fcdb81033c0 2024-05-08T22:35:42.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.255+0000 7fcdb5ffb700 1 -- 172.21.15.62:0/2176800244 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcdac099e20 con 0x7fcdb81033c0 2024-05-08T22:35:42.256 INFO:teuthology.orchestra.run.smithi062.stdout:184683593761 2024-05-08T22:35:42.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.257+0000 7fcdbec0a700 1 -- 172.21.15.62:0/2176800244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcda80a1230 msgr2=0x7fcda80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.257+0000 7fcdbec0a700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcda80a1230 0x7fcda80a36f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcda4005e10 tx=0x7fcda4005d60 comp rx=0 tx=0).stop 2024-05-08T22:35:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.257+0000 7fcdbec0a700 1 -- 172.21.15.62:0/2176800244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb81033c0 msgr2=0x7fcdb81996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.257+0000 7fcdbec0a700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb81033c0 0x7fcdb81996f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e9a0 tx=0x7fcdac00ed60 comp rx=0 tx=0).stop 2024-05-08T22:35:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.258+0000 7fcdbec0a700 1 -- 172.21.15.62:0/2176800244 shutdown_connections 2024-05-08T22:35:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.258+0000 7fcdbec0a700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdb81084a0 0x7fcdb819df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.258+0000 7fcdbec0a700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb81033c0 0x7fcdb81996f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.258+0000 7fcdbec0a700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcda80a1230 0x7fcda80a36f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.258+0000 7fcdbec0a700 1 --2- 172.21.15.62:0/2176800244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb8103d70 0x7fcdb8199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.258+0000 7fcdbec0a700 1 -- 172.21.15.62:0/2176800244 >> 172.21.15.62:0/2176800244 conn(0x7fcdb80fec50 msgr2=0x7fcdb8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.258+0000 7fcdbec0a700 1 -- 172.21.15.62:0/2176800244 shutdown_connections 2024-05-08T22:35:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.258+0000 7fcdbec0a700 1 -- 172.21.15.62:0/2176800244 wait complete. 2024-05-08T22:35:42.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:41 smithi066 ceph-mon[46830]: pgmap v237: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-08T22:35:42.381 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:42.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:41 smithi079 ceph-mon[46738]: pgmap v237: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-08T22:35:42.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.450+0000 7f5c67dd9700 1 -- 172.21.15.62:0/4132613748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c60068bb0 msgr2=0x7f5c60068f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.450+0000 7f5c67dd9700 1 --2- 172.21.15.62:0/4132613748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c60068bb0 0x7f5c60068f90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009990 tx=0x7f5c5c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.450+0000 7f5c67dd9700 1 -- 172.21.15.62:0/4132613748 shutdown_connections 2024-05-08T22:35:42.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.450+0000 7f5c67dd9700 1 --2- 172.21.15.62:0/4132613748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c60106340 0x7f5c6010dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.450+0000 7f5c67dd9700 1 --2- 172.21.15.62:0/4132613748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c60068bb0 0x7f5c60068f90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.450+0000 7f5c67dd9700 1 --2- 172.21.15.62:0/4132613748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c60069560 0x7f5c60105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.450+0000 7f5c67dd9700 1 -- 172.21.15.62:0/4132613748 >> 172.21.15.62:0/4132613748 conn(0x7f5c60075f80 msgr2=0x7f5c60076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:42.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.450+0000 7fcb80ce5700 1 -- 172.21.15.62:0/2250551423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fcb7c110f80 con 0x7fcb7c104ed0 2024-05-08T22:35:42.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.450+0000 7f5c67dd9700 1 -- 172.21.15.62:0/4132613748 shutdown_connections 2024-05-08T22:35:42.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.450+0000 7f5c67dd9700 1 -- 172.21.15.62:0/4132613748 wait complete. 2024-05-08T22:35:42.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.451+0000 7f5c67dd9700 1 Processor -- start 2024-05-08T22:35:42.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.451+0000 7fcb6b7fe700 1 -- 172.21.15.62:0/2250551423 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fcb74025390 con 0x7fcb7c104ed0 2024-05-08T22:35:42.453 INFO:teuthology.orchestra.run.smithi062.stdout:317827579921 2024-05-08T22:35:42.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.451+0000 7f5c67dd9700 1 -- start start 2024-05-08T22:35:42.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.451+0000 7f5c67dd9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c60068bb0 0x7f5c60199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.451+0000 7f5c67dd9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c60069560 0x7f5c60199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.451+0000 7f5c67dd9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c60106340 0x7f5c6019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.451+0000 7f5c67dd9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6019e570 con 0x7f5c60106340 2024-05-08T22:35:42.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.451+0000 7f5c67dd9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6019e6e0 con 0x7f5c60069560 2024-05-08T22:35:42.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.451+0000 7f5c67dd9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6019e8a0 con 0x7f5c60068bb0 2024-05-08T22:35:42.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c66376700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c60106340 0x7f5c6019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c65b75700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c60068bb0 0x7f5c60199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c65374700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c60069560 0x7f5c60199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c66376700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c60106340 0x7f5c6019df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51626/0 (socket says 172.21.15.62:51626) 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c66376700 1 -- 172.21.15.62:0/496249154 learned_addr learned my addr 172.21.15.62:0/496249154 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c65374700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c60069560 0x7f5c60199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.62:54312/0 (socket says 172.21.15.62:54312) 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c66376700 1 -- 172.21.15.62:0/496249154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c60068bb0 msgr2=0x7f5c60199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c66376700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c60068bb0 0x7f5c60199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c66376700 1 -- 172.21.15.62:0/496249154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c60069560 msgr2=0x7f5c60199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c66376700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c60069560 0x7f5c60199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c66376700 1 -- 172.21.15.62:0/496249154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c0095f0 con 0x7f5c60106340 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c66376700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c60106340 0x7f5c6019df60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800ada0 tx=0x7f5c5800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c56ffd700 1 -- 172.21.15.62:0/496249154 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5800f070 con 0x7f5c60106340 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c67dd9700 1 -- 172.21.15.62:0/496249154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6019eb80 con 0x7f5c60106340 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c56ffd700 1 -- 172.21.15.62:0/496249154 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c58007b30 con 0x7f5c60106340 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c56ffd700 1 -- 172.21.15.62:0/496249154 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c58005280 con 0x7f5c60106340 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.452+0000 7f5c67dd9700 1 -- 172.21.15.62:0/496249154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c601a63e0 con 0x7f5c60106340 2024-05-08T22:35:42.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7f5c67dd9700 1 -- 172.21.15.62:0/496249154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c601026f0 con 0x7f5c60106340 2024-05-08T22:35:42.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 -- 172.21.15.62:0/2250551423 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb640a11b0 msgr2=0x7fcb640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb640a11b0 0x7fcb640a3670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009ea0 tx=0x7fcb6c009210 comp rx=0 tx=0).stop 2024-05-08T22:35:42.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 -- 172.21.15.62:0/2250551423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb7c104ed0 msgr2=0x7fcb7c19df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb7c104ed0 0x7fcb7c19df60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400efc0 tx=0x7fcb7400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 -- 172.21.15.62:0/2250551423 shutdown_connections 2024-05-08T22:35:42.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcb640a11b0 0x7fcb640a3670 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb7c104ed0 0x7fcb7c19df60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c100a50 0x7fcb7c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 --2- 172.21.15.62:0/2250551423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb7c101410 0x7fcb7c199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 -- 172.21.15.62:0/2250551423 >> 172.21.15.62:0/2250551423 conn(0x7fcb7c075260 msgr2=0x7fcb7c10b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:42.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 -- 172.21.15.62:0/2250551423 shutdown_connections 2024-05-08T22:35:42.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.454+0000 7fcb80ce5700 1 -- 172.21.15.62:0/2250551423 wait complete. 2024-05-08T22:35:42.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.455+0000 7f5c56ffd700 1 -- 172.21.15.62:0/496249154 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5c5800c8d0 con 0x7f5c60106340 2024-05-08T22:35:42.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.455+0000 7f5c56ffd700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5c4c0a11b0 0x7f5c4c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:42.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.455+0000 7f5c65b75700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5c4c0a11b0 0x7f5c4c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:42.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.455+0000 7f5c56ffd700 1 -- 172.21.15.62:0/496249154 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5c580cbdc0 con 0x7f5c60106340 2024-05-08T22:35:42.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.456+0000 7f5c65b75700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5c4c0a11b0 0x7f5c4c0a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00b5c0 tx=0x7f5c5c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:42.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.460+0000 7f5c56ffd700 1 -- 172.21.15.62:0/496249154 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5c58095630 con 0x7f5c60106340 2024-05-08T22:35:42.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.670+0000 7f5c67dd9700 1 -- 172.21.15.62:0/496249154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f5c6019a9b0 con 0x7f5c60106340 2024-05-08T22:35:42.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.670+0000 7f5c56ffd700 1 -- 172.21.15.62:0/496249154 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c58098c50 con 0x7f5c60106340 2024-05-08T22:35:42.671 INFO:teuthology.orchestra.run.smithi062.stdout:266287972376 2024-05-08T22:35:42.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 -- 172.21.15.62:0/496249154 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5c4c0a11b0 msgr2=0x7f5c4c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5c4c0a11b0 0x7f5c4c0a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00b5c0 tx=0x7f5c5c005fd0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 -- 172.21.15.62:0/496249154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c60106340 msgr2=0x7f5c6019df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:42.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c60106340 0x7f5c6019df60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800ada0 tx=0x7f5c5800c330 comp rx=0 tx=0).stop 2024-05-08T22:35:42.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 -- 172.21.15.62:0/496249154 shutdown_connections 2024-05-08T22:35:42.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c60106340 0x7f5c6019df60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5c60068bb0 0x7f5c60199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5c4c0a11b0 0x7f5c4c0a3670 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 --2- 172.21.15.62:0/496249154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c60069560 0x7f5c60199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:42.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 -- 172.21.15.62:0/496249154 >> 172.21.15.62:0/496249154 conn(0x7f5c60075f80 msgr2=0x7f5c6010c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:42.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 -- 172.21.15.62:0/496249154 shutdown_connections 2024-05-08T22:35:42.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.673+0000 7f5c67dd9700 1 -- 172.21.15.62:0/496249154 wait complete. 2024-05-08T22:35:42.872 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593760 got 184683593761 for osd.6 2024-05-08T22:35:42.872 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:42.955 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:43.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.876+0000 7f7c1c64e700 1 -- 172.21.15.62:0/949262772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14069210 msgr2=0x7f7c14069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:43.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.876+0000 7f7c1c64e700 1 --2- 172.21.15.62:0/949262772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14069210 0x7f7c14069690 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7c080099c0 tx=0x7f7c08009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.876+0000 7f7c1c64e700 1 -- 172.21.15.62:0/949262772 shutdown_connections 2024-05-08T22:35:43.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.876+0000 7f7c1c64e700 1 --2- 172.21.15.62:0/949262772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c14069bd0 0x7f7c1410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.876+0000 7f7c1c64e700 1 --2- 172.21.15.62:0/949262772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c14106fe0 0x7f7c141073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.876+0000 7f7c1c64e700 1 --2- 172.21.15.62:0/949262772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14069210 0x7f7c14069690 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.876+0000 7f7c1c64e700 1 -- 172.21.15.62:0/949262772 >> 172.21.15.62:0/949262772 conn(0x7f7c14076e70 msgr2=0x7f7c14077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:43.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.876+0000 7f7c1c64e700 1 -- 172.21.15.62:0/949262772 shutdown_connections 2024-05-08T22:35:43.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.876+0000 7f7c1c64e700 1 -- 172.21.15.62:0/949262772 wait complete. 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1c64e700 1 Processor -- start 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1c64e700 1 -- start start 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1c64e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c14069210 0x7f7c141972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1c64e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c14069bd0 0x7f7c14197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1c64e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14106fe0 0x7f7c1419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1c64e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1419c1f0 con 0x7f7c14106fe0 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1c64e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1419c360 con 0x7f7c14069bd0 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1c64e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1419c520 con 0x7f7c14069210 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1abeb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14106fe0 0x7f7c1419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1abeb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14106fe0 0x7f7c1419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51634/0 (socket says 172.21.15.62:51634) 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1abeb700 1 -- 172.21.15.62:0/228354523 learned_addr learned my addr 172.21.15.62:0/228354523 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:43.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c1a3ea700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c14069210 0x7f7c141972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.877+0000 7f7c19be9700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c14069bd0 0x7f7c14197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.878+0000 7f7c1abeb700 1 -- 172.21.15.62:0/228354523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c14069210 msgr2=0x7f7c141972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.878+0000 7f7c1abeb700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c14069210 0x7f7c141972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.878+0000 7f7c1abeb700 1 -- 172.21.15.62:0/228354523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c14069bd0 msgr2=0x7f7c14197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.878+0000 7f7c1abeb700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c14069bd0 0x7f7c14197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.878+0000 7f7c1abeb700 1 -- 172.21.15.62:0/228354523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c080095f0 con 0x7f7c14106fe0 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.878+0000 7f7c1abeb700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14106fe0 0x7f7c1419bbe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00d980 tx=0x7f7c0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.878+0000 7f7c137fe700 1 -- 172.21.15.62:0/228354523 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c004a30 con 0x7f7c14106fe0 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.878+0000 7f7c1c64e700 1 -- 172.21.15.62:0/228354523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1419c800 con 0x7f7c14106fe0 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.878+0000 7f7c137fe700 1 -- 172.21.15.62:0/228354523 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c0c0054e0 con 0x7f7c14106fe0 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.878+0000 7f7c137fe700 1 -- 172.21.15.62:0/228354523 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c00fbc0 con 0x7f7c14106fe0 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.878+0000 7f7c1c64e700 1 -- 172.21.15.62:0/228354523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c141a4060 con 0x7f7c14106fe0 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.879+0000 7f7c1c64e700 1 -- 172.21.15.62:0/228354523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c141094e0 con 0x7f7c14106fe0 2024-05-08T22:35:43.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.880+0000 7f7c137fe700 1 -- 172.21.15.62:0/228354523 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7c0c004c20 con 0x7f7c14106fe0 2024-05-08T22:35:43.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.881+0000 7f7c137fe700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7bfc0a11b0 0x7f7bfc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:43.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.881+0000 7f7c1a3ea700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7bfc0a11b0 0x7f7bfc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:43.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.881+0000 7f7c137fe700 1 -- 172.21.15.62:0/228354523 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7c0c0cc680 con 0x7f7c14106fe0 2024-05-08T22:35:43.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.881+0000 7f7c1a3ea700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7bfc0a11b0 0x7f7bfc0a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009550 tx=0x7f7c040091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:43.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:42.885+0000 7f7c137fe700 1 -- 172.21.15.62:0/228354523 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7c0c095e60 con 0x7f7c14106fe0 2024-05-08T22:35:43.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.099+0000 7f7c1c64e700 1 -- 172.21.15.62:0/228354523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7c14198600 con 0x7f7c14106fe0 2024-05-08T22:35:43.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.099+0000 7f7c137fe700 1 -- 172.21.15.62:0/228354523 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c0c099480 con 0x7f7c14106fe0 2024-05-08T22:35:43.099 INFO:teuthology.orchestra.run.smithi062.stdout:154618822694 2024-05-08T22:35:43.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 -- 172.21.15.62:0/228354523 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7bfc0a11b0 msgr2=0x7f7bfc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:43.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7bfc0a11b0 0x7f7bfc0a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009550 tx=0x7f7c040091b0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 -- 172.21.15.62:0/228354523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14106fe0 msgr2=0x7f7c1419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14106fe0 0x7f7c1419bbe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00d980 tx=0x7f7c0c00dc90 comp rx=0 tx=0).stop 2024-05-08T22:35:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 -- 172.21.15.62:0/228354523 shutdown_connections 2024-05-08T22:35:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14106fe0 0x7f7c1419bbe0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c14069210 0x7f7c141972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7bfc0a11b0 0x7f7bfc0a3670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 --2- 172.21.15.62:0/228354523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c14069bd0 0x7f7c14197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 -- 172.21.15.62:0/228354523 >> 172.21.15.62:0/228354523 conn(0x7f7c14076e70 msgr2=0x7f7c140feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 -- 172.21.15.62:0/228354523 shutdown_connections 2024-05-08T22:35:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.101+0000 7f7c1c64e700 1 -- 172.21.15.62:0/228354523 wait complete. 2024-05-08T22:35:43.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:42 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3292169942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T22:35:43.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:42 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2176800244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T22:35:43.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:42 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2250551423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T22:35:43.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:42 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/496249154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T22:35:43.265 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972375 got 266287972376 for osd.9 2024-05-08T22:35:43.265 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:43.299 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215147 got 103079215148 for osd.3 2024-05-08T22:35:43.299 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:43.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:42 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3292169942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T22:35:43.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:42 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2176800244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T22:35:43.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:42 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2250551423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T22:35:43.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:42 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/496249154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T22:35:43.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:42 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3292169942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T22:35:43.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:42 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2176800244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T22:35:43.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:42 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2250551423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T22:35:43.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:42 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/496249154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T22:35:43.425 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579919 got 317827579921 for osd.11 2024-05-08T22:35:43.425 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:43.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.652+0000 7fc987f40700 1 -- 172.21.15.62:0/1502089345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc980107020 msgr2=0x7fc980107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:43.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.652+0000 7fc987f40700 1 --2- 172.21.15.62:0/1502089345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc980107020 0x7fc980107400 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009990 tx=0x7fc97c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.653+0000 7fc987f40700 1 -- 172.21.15.62:0/1502089345 shutdown_connections 2024-05-08T22:35:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.653+0000 7fc987f40700 1 --2- 172.21.15.62:0/1502089345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9800692f0 0x7fc98010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.653+0000 7fc987f40700 1 --2- 172.21.15.62:0/1502089345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc980107020 0x7fc980107400 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.653+0000 7fc987f40700 1 --2- 172.21.15.62:0/1502089345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc980068950 0x7fc980068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.653+0000 7fc987f40700 1 -- 172.21.15.62:0/1502089345 >> 172.21.15.62:0/1502089345 conn(0x7fc980075f80 msgr2=0x7fc980076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.653+0000 7fc987f40700 1 -- 172.21.15.62:0/1502089345 shutdown_connections 2024-05-08T22:35:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.653+0000 7fc987f40700 1 -- 172.21.15.62:0/1502089345 wait complete. 2024-05-08T22:35:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.653+0000 7fc987f40700 1 Processor -- start 2024-05-08T22:35:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.653+0000 7fc987f40700 1 -- start start 2024-05-08T22:35:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.653+0000 7fc987f40700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc980068950 0x7fc9801973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.653+0000 7fc987f40700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9800692f0 0x7fc9801978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc987f40700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc980107020 0x7fc98019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:43.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc987f40700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98019c240 con 0x7fc980068950 2024-05-08T22:35:43.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc987f40700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98019c3b0 con 0x7fc9800692f0 2024-05-08T22:35:43.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc987f40700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98019c570 con 0x7fc980107020 2024-05-08T22:35:43.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc985cdc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc980068950 0x7fc9801973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:43.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc985cdc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc980068950 0x7fc9801973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51658/0 (socket says 172.21.15.62:51658) 2024-05-08T22:35:43.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc985cdc700 1 -- 172.21.15.62:0/488843432 learned_addr learned my addr 172.21.15.62:0/488843432 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:43.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc9854db700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9800692f0 0x7fc9801978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:43.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc9864dd700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc980107020 0x7fc98019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc985cdc700 1 -- 172.21.15.62:0/488843432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc980107020 msgr2=0x7fc98019bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc985cdc700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc980107020 0x7fc98019bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc985cdc700 1 -- 172.21.15.62:0/488843432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9800692f0 msgr2=0x7fc9801978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc985cdc700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9800692f0 0x7fc9801978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc985cdc700 1 -- 172.21.15.62:0/488843432 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c0095f0 con 0x7fc980068950 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc985cdc700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc980068950 0x7fc9801973b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc97c000c00 tx=0x7fc97c00f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc976ffd700 1 -- 172.21.15.62:0/488843432 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c01d070 con 0x7fc980068950 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc976ffd700 1 -- 172.21.15.62:0/488843432 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc97c00fd30 con 0x7fc980068950 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc976ffd700 1 -- 172.21.15.62:0/488843432 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c017630 con 0x7fc980068950 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc987f40700 1 -- 172.21.15.62:0/488843432 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98019c7f0 con 0x7fc980068950 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.654+0000 7fc987f40700 1 -- 172.21.15.62:0/488843432 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9801a3ee0 con 0x7fc980068950 2024-05-08T22:35:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.655+0000 7fc987f40700 1 -- 172.21.15.62:0/488843432 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc980109540 con 0x7fc980068950 2024-05-08T22:35:43.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.657+0000 7fc976ffd700 1 -- 172.21.15.62:0/488843432 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc97c004830 con 0x7fc980068950 2024-05-08T22:35:43.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.658+0000 7fc976ffd700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc96c0a11b0 0x7fc96c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:43.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.658+0000 7fc976ffd700 1 -- 172.21.15.62:0/488843432 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc97c0cdc40 con 0x7fc980068950 2024-05-08T22:35:43.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.658+0000 7fc9854db700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc96c0a11b0 0x7fc96c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:43.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.658+0000 7fc9854db700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc96c0a11b0 0x7fc96c0a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9800766c0 tx=0x7fc97000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:43.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.661+0000 7fc976ffd700 1 -- 172.21.15.62:0/488843432 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc97c0974b0 con 0x7fc980068950 2024-05-08T22:35:43.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.873+0000 7fc987f40700 1 -- 172.21.15.62:0/488843432 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc9801026d0 con 0x7fc980068950 2024-05-08T22:35:43.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.873+0000 7fc976ffd700 1 -- 172.21.15.62:0/488843432 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc97c09aad0 con 0x7fc980068950 2024-05-08T22:35:43.874 INFO:teuthology.orchestra.run.smithi062.stdout:206158430239 2024-05-08T22:35:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.875+0000 7fc987f40700 1 -- 172.21.15.62:0/488843432 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc96c0a11b0 msgr2=0x7fc96c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.875+0000 7fc987f40700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc96c0a11b0 0x7fc96c0a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9800766c0 tx=0x7fc97000a300 comp rx=0 tx=0).stop 2024-05-08T22:35:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.875+0000 7fc987f40700 1 -- 172.21.15.62:0/488843432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc980068950 msgr2=0x7fc9801973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.875+0000 7fc987f40700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc980068950 0x7fc9801973b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc97c000c00 tx=0x7fc97c00f7d0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.876+0000 7fc987f40700 1 -- 172.21.15.62:0/488843432 shutdown_connections 2024-05-08T22:35:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.876+0000 7fc987f40700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc980107020 0x7fc98019bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.876+0000 7fc987f40700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc980068950 0x7fc9801973b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.876+0000 7fc987f40700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc96c0a11b0 0x7fc96c0a3670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.876+0000 7fc987f40700 1 --2- 172.21.15.62:0/488843432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9800692f0 0x7fc9801978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:43.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.876+0000 7fc987f40700 1 -- 172.21.15.62:0/488843432 >> 172.21.15.62:0/488843432 conn(0x7fc980075f80 msgr2=0x7fc9800fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:43.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.876+0000 7fc987f40700 1 -- 172.21.15.62:0/488843432 shutdown_connections 2024-05-08T22:35:43.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:43.876+0000 7fc987f40700 1 -- 172.21.15.62:0/488843432 wait complete. 2024-05-08T22:35:44.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:43 smithi062 ceph-mon[42571]: pgmap v238: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-08T22:35:44.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:43 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/228354523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T22:35:44.023 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:43 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/488843432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T22:35:44.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:43 smithi066 ceph-mon[46830]: pgmap v238: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-08T22:35:44.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:43 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/228354523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T22:35:44.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:43 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/488843432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T22:35:44.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:43 smithi079 ceph-mon[46738]: pgmap v238: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-08T22:35:44.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:43 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/228354523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T22:35:44.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:43 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/488843432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T22:35:44.400 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822692 got 154618822694 for osd.5 2024-05-08T22:35:44.400 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:44.507 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430237 got 206158430239 for osd.7 2024-05-08T22:35:44.507 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:44.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.902+0000 7f676840b700 1 -- 172.21.15.62:0/1713173073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67600ffed0 msgr2=0x7f6760100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:44.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.902+0000 7f676840b700 1 --2- 172.21.15.62:0/1713173073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67600ffed0 0x7f6760100350 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67500099c0 tx=0x7f6750009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.902+0000 7f676840b700 1 -- 172.21.15.62:0/1713173073 shutdown_connections 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.902+0000 7f676840b700 1 --2- 172.21.15.62:0/1713173073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760100890 0x7f676010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.902+0000 7f676840b700 1 --2- 172.21.15.62:0/1713173073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760105ee0 0x7f67601062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.902+0000 7f676840b700 1 --2- 172.21.15.62:0/1713173073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67600ffed0 0x7f6760100350 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.902+0000 7f676840b700 1 -- 172.21.15.62:0/1713173073 >> 172.21.15.62:0/1713173073 conn(0x7f6760075f80 msgr2=0x7f6760076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.903+0000 7f676840b700 1 -- 172.21.15.62:0/1713173073 shutdown_connections 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.903+0000 7f676840b700 1 -- 172.21.15.62:0/1713173073 wait complete. 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.903+0000 7f676840b700 1 Processor -- start 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.903+0000 7f676840b700 1 -- start start 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.903+0000 7f676840b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67600ffed0 0x7f67601981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.903+0000 7f676840b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760100890 0x7f67601986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:44.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.903+0000 7f676840b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760105ee0 0x7f6760198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f676840b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67601a3490 con 0x7f67600ffed0 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f676840b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67601a3600 con 0x7f6760100890 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f676840b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67601a37c0 con 0x7f6760105ee0 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67661a7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67600ffed0 0x7f67601981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67661a7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67600ffed0 0x7f67601981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51670/0 (socket says 172.21.15.62:51670) 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67661a7700 1 -- 172.21.15.62:0/3244381635 learned_addr learned my addr 172.21.15.62:0/3244381635 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67669a8700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760105ee0 0x7f6760198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67659a6700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760100890 0x7f67601986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67661a7700 1 -- 172.21.15.62:0/3244381635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760105ee0 msgr2=0x7f6760198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67661a7700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760105ee0 0x7f6760198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67661a7700 1 -- 172.21.15.62:0/3244381635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760100890 msgr2=0x7f67601986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67661a7700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760100890 0x7f67601986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67661a7700 1 -- 172.21.15.62:0/3244381635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67500095f0 con 0x7f67600ffed0 2024-05-08T22:35:44.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67659a6700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760100890 0x7f67601986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:44.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67661a7700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67600ffed0 0x7f67601981a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f675c00e9a0 tx=0x7f675c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:44.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67577fe700 1 -- 172.21.15.62:0/3244381635 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c009790 con 0x7f67600ffed0 2024-05-08T22:35:44.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67577fe700 1 -- 172.21.15.62:0/3244381635 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f675c00cbd0 con 0x7f67600ffed0 2024-05-08T22:35:44.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f676840b700 1 -- 172.21.15.62:0/3244381635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67601a3aa0 con 0x7f67600ffed0 2024-05-08T22:35:44.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.904+0000 7f67577fe700 1 -- 172.21.15.62:0/3244381635 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c018680 con 0x7f67600ffed0 2024-05-08T22:35:44.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.905+0000 7f676840b700 1 -- 172.21.15.62:0/3244381635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67601a3f70 con 0x7f67600ffed0 2024-05-08T22:35:44.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.906+0000 7f676840b700 1 -- 172.21.15.62:0/3244381635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67601083e0 con 0x7f67600ffed0 2024-05-08T22:35:44.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.907+0000 7f67577fe700 1 -- 172.21.15.62:0/3244381635 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f675c00cd40 con 0x7f67600ffed0 2024-05-08T22:35:44.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.908+0000 7f67577fe700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a11b0 0x7f674c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:44.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.908+0000 7f67659a6700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a11b0 0x7f674c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:44.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.908+0000 7f67577fe700 1 -- 172.21.15.62:0/3244381635 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f675c014070 con 0x7f67600ffed0 2024-05-08T22:35:44.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.909+0000 7f67659a6700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a11b0 0x7f674c0a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6750009fd0 tx=0x7f6750005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:44.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.912+0000 7f67577fe700 1 -- 172.21.15.62:0/3244381635 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f675c052d60 con 0x7f67600ffed0 2024-05-08T22:35:44.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.942+0000 7f72e8108700 1 -- 172.21.15.62:0/1882284040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e0107020 msgr2=0x7f72e0107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:44.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.942+0000 7f72e8108700 1 --2- 172.21.15.62:0/1882284040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e0107020 0x7f72e0107400 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009990 tx=0x7f72dc009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.942+0000 7f72e8108700 1 -- 172.21.15.62:0/1882284040 shutdown_connections 2024-05-08T22:35:44.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.942+0000 7f72e8108700 1 --2- 172.21.15.62:0/1882284040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72e00692f0 0x7f72e010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.942+0000 7f72e8108700 1 --2- 172.21.15.62:0/1882284040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e0107020 0x7f72e0107400 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.942+0000 7f72e8108700 1 --2- 172.21.15.62:0/1882284040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e0068950 0x7f72e0068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.942+0000 7f72e8108700 1 -- 172.21.15.62:0/1882284040 >> 172.21.15.62:0/1882284040 conn(0x7f72e0075f80 msgr2=0x7f72e0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:44.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.942+0000 7f72e8108700 1 -- 172.21.15.62:0/1882284040 shutdown_connections 2024-05-08T22:35:44.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.942+0000 7f72e8108700 1 -- 172.21.15.62:0/1882284040 wait complete. 2024-05-08T22:35:44.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.942+0000 7f72e8108700 1 Processor -- start 2024-05-08T22:35:44.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e8108700 1 -- start start 2024-05-08T22:35:44.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e8108700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e0068950 0x7f72e00fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:44.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e8108700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72e00692f0 0x7f72e01004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:44.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e8108700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e0107020 0x7f72e0100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:44.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e8108700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e01a5730 con 0x7f72e0107020 2024-05-08T22:35:44.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e8108700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e01a58a0 con 0x7f72e00692f0 2024-05-08T22:35:44.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e8108700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e01a5a60 con 0x7f72e0068950 2024-05-08T22:35:44.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e66a5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e0107020 0x7f72e0100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:44.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e66a5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e0107020 0x7f72e0100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51694/0 (socket says 172.21.15.62:51694) 2024-05-08T22:35:44.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e66a5700 1 -- 172.21.15.62:0/2566479765 learned_addr learned my addr 172.21.15.62:0/2566479765 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:44.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e5ea4700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e0068950 0x7f72e00fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:44.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e56a3700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72e00692f0 0x7f72e01004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:44.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e66a5700 1 -- 172.21.15.62:0/2566479765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e0068950 msgr2=0x7f72e00fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:44.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e66a5700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e0068950 0x7f72e00fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e66a5700 1 -- 172.21.15.62:0/2566479765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72e00692f0 msgr2=0x7f72e01004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:44.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e66a5700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72e00692f0 0x7f72e01004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:44.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e66a5700 1 -- 172.21.15.62:0/2566479765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc0095f0 con 0x7f72e0107020 2024-05-08T22:35:44.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e56a3700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72e00692f0 0x7f72e01004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:44.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72e66a5700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e0107020 0x7f72e0100a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f72d8007ee0 tx=0x7f72d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:44.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.943+0000 7f72d6ffd700 1 -- 172.21.15.62:0/2566479765 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d800ddd0 con 0x7f72e0107020 2024-05-08T22:35:44.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.944+0000 7f72d6ffd700 1 -- 172.21.15.62:0/2566479765 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72d800f040 con 0x7f72e0107020 2024-05-08T22:35:44.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.944+0000 7f72e8108700 1 -- 172.21.15.62:0/2566479765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e01a5d40 con 0x7f72e0107020 2024-05-08T22:35:44.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.944+0000 7f72d6ffd700 1 -- 172.21.15.62:0/2566479765 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d80048e0 con 0x7f72e0107020 2024-05-08T22:35:44.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.944+0000 7f72e8108700 1 -- 172.21.15.62:0/2566479765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e01a61f0 con 0x7f72e0107020 2024-05-08T22:35:44.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.945+0000 7f72d4ff9700 1 -- 172.21.15.62:0/2566479765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e01026d0 con 0x7f72e0107020 2024-05-08T22:35:44.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.946+0000 7f72d6ffd700 1 -- 172.21.15.62:0/2566479765 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f72d8003a00 con 0x7f72e0107020 2024-05-08T22:35:44.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.947+0000 7f72d6ffd700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f72cc0a11b0 0x7f72cc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:44.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.947+0000 7f72e5ea4700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f72cc0a11b0 0x7f72cc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:44.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.947+0000 7f72d6ffd700 1 -- 172.21.15.62:0/2566479765 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72d80cca90 con 0x7f72e0107020 2024-05-08T22:35:44.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.947+0000 7f72e5ea4700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f72cc0a11b0 0x7f72cc0a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0098e0 tx=0x7f72dc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:44.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:44.950+0000 7f72d6ffd700 1 -- 172.21.15.62:0/2566479765 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f72d8096170 con 0x7f72e0107020 2024-05-08T22:35:45.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.118+0000 7f676840b700 1 -- 172.21.15.62:0/3244381635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6760101590 con 0x7f67600ffed0 2024-05-08T22:35:45.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.119+0000 7f67577fe700 1 -- 172.21.15.62:0/3244381635 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6760101590 con 0x7f67600ffed0 2024-05-08T22:35:45.119 INFO:teuthology.orchestra.run.smithi062.stdout:55834574901 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 -- 172.21.15.62:0/3244381635 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a11b0 msgr2=0x7f674c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a11b0 0x7f674c0a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6750009fd0 tx=0x7f6750005a90 comp rx=0 tx=0).stop 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 -- 172.21.15.62:0/3244381635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67600ffed0 msgr2=0x7f67601981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67600ffed0 0x7f67601981a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f675c00e9a0 tx=0x7f675c00ecb0 comp rx=0 tx=0).stop 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 -- 172.21.15.62:0/3244381635 shutdown_connections 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760105ee0 0x7f6760198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67600ffed0 0x7f67601981a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a11b0 0x7f674c0a3670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 --2- 172.21.15.62:0/3244381635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760100890 0x7f67601986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 -- 172.21.15.62:0/3244381635 >> 172.21.15.62:0/3244381635 conn(0x7f6760075f80 msgr2=0x7f67600fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 -- 172.21.15.62:0/3244381635 shutdown_connections 2024-05-08T22:35:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.121+0000 7f676840b700 1 -- 172.21.15.62:0/3244381635 wait complete. 2024-05-08T22:35:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.157+0000 7f72d4ff9700 1 -- 172.21.15.62:0/2566479765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f72e004ec60 con 0x7f72e0107020 2024-05-08T22:35:45.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.158+0000 7f72d6ffd700 1 -- 172.21.15.62:0/2566479765 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f72d8023090 con 0x7f72e0107020 2024-05-08T22:35:45.158 INFO:teuthology.orchestra.run.smithi062.stdout:34359738425 2024-05-08T22:35:45.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 -- 172.21.15.62:0/2566479765 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f72cc0a11b0 msgr2=0x7f72cc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:45.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f72cc0a11b0 0x7f72cc0a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0098e0 tx=0x7f72dc005960 comp rx=0 tx=0).stop 2024-05-08T22:35:45.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 -- 172.21.15.62:0/2566479765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e0107020 msgr2=0x7f72e0100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:45.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e0107020 0x7f72e0100a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f72d8007ee0 tx=0x7f72d800d3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:45.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 -- 172.21.15.62:0/2566479765 shutdown_connections 2024-05-08T22:35:45.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e0107020 0x7f72e0100a20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:45.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72e0068950 0x7f72e00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:45.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f72cc0a11b0 0x7f72cc0a3670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:45.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 --2- 172.21.15.62:0/2566479765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72e00692f0 0x7f72e01004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:45.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 -- 172.21.15.62:0/2566479765 >> 172.21.15.62:0/2566479765 conn(0x7f72e0075f80 msgr2=0x7f72e00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:45.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 -- 172.21.15.62:0/2566479765 shutdown_connections 2024-05-08T22:35:45.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:45.160+0000 7f72d4ff9700 1 -- 172.21.15.62:0/2566479765 wait complete. 2024-05-08T22:35:45.653 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574899 got 55834574901 for osd.1 2024-05-08T22:35:45.653 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:45.654 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738423 got 34359738425 for osd.0 2024-05-08T22:35:45.654 DEBUG:teuthology.parallel:result is None 2024-05-08T22:35:45.654 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T22:35:45.654 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T22:35:45.885 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:46.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:45 smithi062 ceph-mon[42571]: pgmap v239: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T22:35:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:45 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3244381635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T22:35:46.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:45 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2566479765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T22:35:46.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:45 smithi066 ceph-mon[46830]: pgmap v239: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T22:35:46.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:45 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3244381635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T22:35:46.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:45 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2566479765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T22:35:46.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:45 smithi079 ceph-mon[46738]: pgmap v239: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T22:35:46.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:45 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3244381635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T22:35:46.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:45 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2566479765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T22:35:46.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.779+0000 7fdc533a0700 1 -- 172.21.15.62:0/4161627970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c105f00 msgr2=0x7fdc4c1062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.779+0000 7fdc533a0700 1 --2- 172.21.15.62:0/4161627970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c105f00 0x7fdc4c1062e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009990 tx=0x7fdc48009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.780+0000 7fdc533a0700 1 -- 172.21.15.62:0/4161627970 shutdown_connections 2024-05-08T22:35:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.780+0000 7fdc533a0700 1 --2- 172.21.15.62:0/4161627970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc4c100890 0x7fdc4c10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.780+0000 7fdc533a0700 1 --2- 172.21.15.62:0/4161627970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c105f00 0x7fdc4c1062e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.780+0000 7fdc533a0700 1 --2- 172.21.15.62:0/4161627970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc4c0ffed0 0x7fdc4c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.780+0000 7fdc533a0700 1 -- 172.21.15.62:0/4161627970 >> 172.21.15.62:0/4161627970 conn(0x7fdc4c075f80 msgr2=0x7fdc4c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.780+0000 7fdc533a0700 1 -- 172.21.15.62:0/4161627970 shutdown_connections 2024-05-08T22:35:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.780+0000 7fdc533a0700 1 -- 172.21.15.62:0/4161627970 wait complete. 2024-05-08T22:35:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.780+0000 7fdc533a0700 1 Processor -- start 2024-05-08T22:35:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.780+0000 7fdc533a0700 1 -- start start 2024-05-08T22:35:46.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.780+0000 7fdc533a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c0ffed0 0x7fdc4c101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.780+0000 7fdc533a0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc4c100890 0x7fdc4c102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc533a0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc4c105f00 0x7fdc4c1029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc533a0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1a5780 con 0x7fdc4c0ffed0 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc533a0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1a58f0 con 0x7fdc4c105f00 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc533a0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1a5ab0 con 0x7fdc4c100890 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc5113c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c0ffed0 0x7fdc4c101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc5113c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c0ffed0 0x7fdc4c101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44002/0 (socket says 172.21.15.62:44002) 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc5113c700 1 -- 172.21.15.62:0/2836626192 learned_addr learned my addr 172.21.15.62:0/2836626192 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc5093b700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc4c100890 0x7fdc4c102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc5193d700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc4c105f00 0x7fdc4c1029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc5113c700 1 -- 172.21.15.62:0/2836626192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc4c100890 msgr2=0x7fdc4c102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:46.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc5113c700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc4c100890 0x7fdc4c102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc5113c700 1 -- 172.21.15.62:0/2836626192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc4c105f00 msgr2=0x7fdc4c1029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc5113c700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc4c105f00 0x7fdc4c1029d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc5113c700 1 -- 172.21.15.62:0/2836626192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc480095f0 con 0x7fdc4c0ffed0 2024-05-08T22:35:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc5113c700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c0ffed0 0x7fdc4c101f50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdc480098e0 tx=0x7fdc4800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc427fc700 1 -- 172.21.15.62:0/2836626192 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4801d070 con 0x7fdc4c0ffed0 2024-05-08T22:35:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc427fc700 1 -- 172.21.15.62:0/2836626192 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc48022470 con 0x7fdc4c0ffed0 2024-05-08T22:35:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc533a0700 1 -- 172.21.15.62:0/2836626192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c1a5d30 con 0x7fdc4c0ffed0 2024-05-08T22:35:46.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc427fc700 1 -- 172.21.15.62:0/2836626192 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48017650 con 0x7fdc4c0ffed0 2024-05-08T22:35:46.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.781+0000 7fdc533a0700 1 -- 172.21.15.62:0/2836626192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c1a61a0 con 0x7fdc4c0ffed0 2024-05-08T22:35:46.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.782+0000 7fdc533a0700 1 -- 172.21.15.62:0/2836626192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4c1015b0 con 0x7fdc4c0ffed0 2024-05-08T22:35:46.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.784+0000 7fdc427fc700 1 -- 172.21.15.62:0/2836626192 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdc480225e0 con 0x7fdc4c0ffed0 2024-05-08T22:35:46.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.784+0000 7fdc427fc700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc380a1230 0x7fdc380a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:46.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.784+0000 7fdc5093b700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc380a1230 0x7fdc380a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:46.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.784+0000 7fdc427fc700 1 -- 172.21.15.62:0/2836626192 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdc480cdbc0 con 0x7fdc4c0ffed0 2024-05-08T22:35:46.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.785+0000 7fdc5093b700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc380a1230 0x7fdc380a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c103630 tx=0x7fdc3c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:46.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.788+0000 7fdc427fc700 1 -- 172.21.15.62:0/2836626192 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdc48097430 con 0x7fdc4c0ffed0 2024-05-08T22:35:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.987+0000 7fdc533a0700 1 -- 172.21.15.62:0/2836626192 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc4c10fdc0 con 0x7fdc380a1230 2024-05-08T22:35:46.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:46.997+0000 7fdc427fc700 1 -- 172.21.15.62:0/2836626192 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340981 (secure 0 0 0) 0x7fdc4c10fdc0 con 0x7fdc380a1230 2024-05-08T22:35:47.002 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:47.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 -- 172.21.15.62:0/2836626192 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc380a1230 msgr2=0x7fdc380a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:47.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc380a1230 0x7fdc380a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c103630 tx=0x7fdc3c00b410 comp rx=0 tx=0).stop 2024-05-08T22:35:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 -- 172.21.15.62:0/2836626192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c0ffed0 msgr2=0x7fdc4c101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c0ffed0 0x7fdc4c101f50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdc480098e0 tx=0x7fdc4800f670 comp rx=0 tx=0).stop 2024-05-08T22:35:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 -- 172.21.15.62:0/2836626192 shutdown_connections 2024-05-08T22:35:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc380a1230 0x7fdc380a36f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc4c105f00 0x7fdc4c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c0ffed0 0x7fdc4c101f50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 --2- 172.21.15.62:0/2836626192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc4c100890 0x7fdc4c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 -- 172.21.15.62:0/2836626192 >> 172.21.15.62:0/2836626192 conn(0x7fdc4c075f80 msgr2=0x7fdc4c104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 -- 172.21.15.62:0/2836626192 shutdown_connections 2024-05-08T22:35:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:47.001+0000 7fdc533a0700 1 -- 172.21.15.62:0/2836626192 wait complete. 2024-05-08T22:35:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-05-08T22:35:47.363 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-05-08T22:35:46.023516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657037,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":444,"num_read_kb":342,"num_write":413,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381524,"kb_used_data":12276,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124560300,"statfs":{"total":1151940427776,"available":1151549747200,"internally_reserved":0,"allocated":12570624,"data_stored":6882127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":377992518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1138,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":56,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003255"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.840865+0000","last_change":"2024-05-08T22:35:09.840453+0000","last_active":"2024-05-08T22:35:09.840865+0000","last_peered":"2024-05-08T22:35:09.840865+0000","last_clean":"2024-05-08T22:35:09.840865+0000","last_became_active":"2024-05-08T22:35:09.840093+0000","last_became_peered":"2024-05-08T22:35:09.840093+0000","last_unstale":"2024-05-08T22:35:09.840865+0000","last_undegraded":"2024-05-08T22:35:09.840865+0000","last_fullsized":"2024-05-08T22:35:09.840865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:13.265286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840996+0000","last_change":"2024-05-08T22:35:09.832735+0000","last_active":"2024-05-08T22:35:11.840996+0000","last_peered":"2024-05-08T22:35:11.840996+0000","last_clean":"2024-05-08T22:35:11.840996+0000","last_became_active":"2024-05-08T22:35:09.832419+0000","last_became_peered":"2024-05-08T22:35:09.832419+0000","last_unstale":"2024-05-08T22:35:11.840996+0000","last_undegraded":"2024-05-08T22:35:11.840996+0000","last_fullsized":"2024-05-08T22:35:11.840996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:19.711398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841216+0000","last_change":"2024-05-08T22:35:09.835604+0000","last_active":"2024-05-08T22:35:11.841216+0000","last_peered":"2024-05-08T22:35:11.841216+0000","last_clean":"2024-05-08T22:35:11.841216+0000","last_became_active":"2024-05-08T22:35:09.835235+0000","last_became_peered":"2024-05-08T22:35:09.835235+0000","last_unstale":"2024-05-08T22:35:11.841216+0000","last_undegraded":"2024-05-08T22:35:11.841216+0000","last_fullsized":"2024-05-08T22:35:11.841216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:35:10.722947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.831424+0000","last_change":"2024-05-08T22:35:09.824876+0000","last_active":"2024-05-08T22:35:09.831424+0000","last_peered":"2024-05-08T22:35:09.831424+0000","last_clean":"2024-05-08T22:35:09.831424+0000","last_became_active":"2024-05-08T22:35:09.823494+0000","last_became_peered":"2024-05-08T22:35:09.823494+0000","last_unstale":"2024-05-08T22:35:09.831424+0000","last_undegraded":"2024-05-08T22:35:09.831424+0000","last_fullsized":"2024-05-08T22:35:09.831424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:20.099947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.828965+0000","last_change":"2024-05-08T22:35:09.825647+0000","last_active":"2024-05-08T22:35:09.828965+0000","last_peered":"2024-05-08T22:35:09.828965+0000","last_clean":"2024-05-08T22:35:09.828965+0000","last_became_active":"2024-05-08T22:35:09.825322+0000","last_became_peered":"2024-05-08T22:35:09.825322+0000","last_unstale":"2024-05-08T22:35:09.828965+0000","last_undegraded":"2024-05-08T22:35:09.828965+0000","last_fullsized":"2024-05-08T22:35:09.828965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:29.080541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604425+0000","last_change":"2024-05-08T22:35:09.841894+0000","last_active":"2024-05-08T22:35:11.604425+0000","last_peered":"2024-05-08T22:35:11.604425+0000","last_clean":"2024-05-08T22:35:11.604425+0000","last_became_active":"2024-05-08T22:35:09.841429+0000","last_became_peered":"2024-05-08T22:35:09.841429+0000","last_unstale":"2024-05-08T22:35:11.604425+0000","last_undegraded":"2024-05-08T22:35:11.604425+0000","last_fullsized":"2024-05-08T22:35:11.604425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:50.859186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.622990+0000","last_change":"2024-05-08T22:35:09.834010+0000","last_active":"2024-05-08T22:35:11.622990+0000","last_peered":"2024-05-08T22:35:11.622990+0000","last_clean":"2024-05-08T22:35:11.622990+0000","last_became_active":"2024-05-08T22:35:09.833633+0000","last_became_peered":"2024-05-08T22:35:09.833633+0000","last_unstale":"2024-05-08T22:35:11.622990+0000","last_undegraded":"2024-05-08T22:35:11.622990+0000","last_fullsized":"2024-05-08T22:35:11.622990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:28.854383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825479+0000","last_change":"2024-05-08T22:35:09.820817+0000","last_active":"2024-05-08T22:35:11.825479+0000","last_peered":"2024-05-08T22:35:11.825479+0000","last_clean":"2024-05-08T22:35:11.825479+0000","last_became_active":"2024-05-08T22:35:09.820521+0000","last_became_peered":"2024-05-08T22:35:09.820521+0000","last_unstale":"2024-05-08T22:35:11.825479+0000","last_undegraded":"2024-05-08T22:35:11.825479+0000","last_fullsized":"2024-05-08T22:35:11.825479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:17.312794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812985+0000","last_change":"2024-05-08T22:35:09.838821+0000","last_active":"2024-05-08T22:35:10.812985+0000","last_peered":"2024-05-08T22:35:10.812985+0000","last_clean":"2024-05-08T22:35:10.812985+0000","last_became_active":"2024-05-08T22:35:09.836588+0000","last_became_peered":"2024-05-08T22:35:09.836588+0000","last_unstale":"2024-05-08T22:35:10.812985+0000","last_undegraded":"2024-05-08T22:35:10.812985+0000","last_fullsized":"2024-05-08T22:35:10.812985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:52.884827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604696+0000","last_change":"2024-05-08T22:35:09.836222+0000","last_active":"2024-05-08T22:35:11.604696+0000","last_peered":"2024-05-08T22:35:11.604696+0000","last_clean":"2024-05-08T22:35:11.604696+0000","last_became_active":"2024-05-08T22:35:09.835810+0000","last_became_peered":"2024-05-08T22:35:09.835810+0000","last_unstale":"2024-05-08T22:35:11.604696+0000","last_undegraded":"2024-05-08T22:35:11.604696+0000","last_fullsized":"2024-05-08T22:35:11.604696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:38:14.075576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623061+0000","last_change":"2024-05-08T22:35:09.835422+0000","last_active":"2024-05-08T22:35:11.623061+0000","last_peered":"2024-05-08T22:35:11.623061+0000","last_clean":"2024-05-08T22:35:11.623061+0000","last_became_active":"2024-05-08T22:35:09.835100+0000","last_became_peered":"2024-05-08T22:35:09.835100+0000","last_unstale":"2024-05-08T22:35:11.623061+0000","last_undegraded":"2024-05-08T22:35:11.623061+0000","last_fullsized":"2024-05-08T22:35:11.623061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:12.099286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.832873+0000","last_change":"2024-05-08T22:35:09.827007+0000","last_active":"2024-05-08T22:35:09.832873+0000","last_peered":"2024-05-08T22:35:09.832873+0000","last_clean":"2024-05-08T22:35:09.832873+0000","last_became_active":"2024-05-08T22:35:09.826639+0000","last_became_peered":"2024-05-08T22:35:09.826639+0000","last_unstale":"2024-05-08T22:35:09.832873+0000","last_undegraded":"2024-05-08T22:35:09.832873+0000","last_fullsized":"2024-05-08T22:35:09.832873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:39.999049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470041+0000","last_change":"2024-05-08T22:35:09.828507+0000","last_active":"2024-05-08T22:35:10.470041+0000","last_peered":"2024-05-08T22:35:10.470041+0000","last_clean":"2024-05-08T22:35:10.470041+0000","last_became_active":"2024-05-08T22:35:09.828076+0000","last_became_peered":"2024-05-08T22:35:09.828076+0000","last_unstale":"2024-05-08T22:35:10.470041+0000","last_undegraded":"2024-05-08T22:35:10.470041+0000","last_fullsized":"2024-05-08T22:35:10.470041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:33.475538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840770+0000","last_change":"2024-05-08T22:35:09.820506+0000","last_active":"2024-05-08T22:35:11.840770+0000","last_peered":"2024-05-08T22:35:11.840770+0000","last_clean":"2024-05-08T22:35:11.840770+0000","last_became_active":"2024-05-08T22:35:09.820208+0000","last_became_peered":"2024-05-08T22:35:09.820208+0000","last_unstale":"2024-05-08T22:35:11.840770+0000","last_undegraded":"2024-05-08T22:35:11.840770+0000","last_fullsized":"2024-05-08T22:35:11.840770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:47:00.753509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840848+0000","last_change":"2024-05-08T22:35:09.837171+0000","last_active":"2024-05-08T22:35:11.840848+0000","last_peered":"2024-05-08T22:35:11.840848+0000","last_clean":"2024-05-08T22:35:11.840848+0000","last_became_active":"2024-05-08T22:35:09.836290+0000","last_became_peered":"2024-05-08T22:35:09.836290+0000","last_unstale":"2024-05-08T22:35:11.840848+0000","last_undegraded":"2024-05-08T22:35:11.840848+0000","last_fullsized":"2024-05-08T22:35:11.840848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:13.029182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812403+0000","last_change":"2024-05-08T22:35:09.836148+0000","last_active":"2024-05-08T22:35:10.812403+0000","last_peered":"2024-05-08T22:35:10.812403+0000","last_clean":"2024-05-08T22:35:10.812403+0000","last_became_active":"2024-05-08T22:35:09.835767+0000","last_became_peered":"2024-05-08T22:35:09.835767+0000","last_unstale":"2024-05-08T22:35:10.812403+0000","last_undegraded":"2024-05-08T22:35:10.812403+0000","last_fullsized":"2024-05-08T22:35:10.812403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:23.343495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.841116+0000","last_change":"2024-05-08T22:35:09.840558+0000","last_active":"2024-05-08T22:35:09.841116+0000","last_peered":"2024-05-08T22:35:09.841116+0000","last_clean":"2024-05-08T22:35:09.841116+0000","last_became_active":"2024-05-08T22:35:09.840080+0000","last_became_peered":"2024-05-08T22:35:09.840080+0000","last_unstale":"2024-05-08T22:35:09.841116+0000","last_undegraded":"2024-05-08T22:35:09.841116+0000","last_fullsized":"2024-05-08T22:35:09.841116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:32.081217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.842689+0000","last_change":"2024-05-08T22:35:09.831151+0000","last_active":"2024-05-08T22:35:11.842689+0000","last_peered":"2024-05-08T22:35:11.842689+0000","last_clean":"2024-05-08T22:35:11.842689+0000","last_became_active":"2024-05-08T22:35:09.830693+0000","last_became_peered":"2024-05-08T22:35:09.830693+0000","last_unstale":"2024-05-08T22:35:11.842689+0000","last_undegraded":"2024-05-08T22:35:11.842689+0000","last_fullsized":"2024-05-08T22:35:11.842689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:15.060787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841080+0000","last_change":"2024-05-08T22:35:09.830620+0000","last_active":"2024-05-08T22:35:11.841080+0000","last_peered":"2024-05-08T22:35:11.841080+0000","last_clean":"2024-05-08T22:35:11.841080+0000","last_became_active":"2024-05-08T22:35:09.830289+0000","last_became_peered":"2024-05-08T22:35:09.830289+0000","last_unstale":"2024-05-08T22:35:11.841080+0000","last_undegraded":"2024-05-08T22:35:11.841080+0000","last_fullsized":"2024-05-08T22:35:11.841080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:49:15.181957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":54,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:42.328576+0000","last_change":"2024-05-08T22:35:09.391570+0000","last_active":"2024-05-08T22:35:42.328576+0000","last_peered":"2024-05-08T22:35:42.328576+0000","last_clean":"2024-05-08T22:35:42.328576+0000","last_became_active":"2024-05-08T22:35:09.391234+0000","last_became_peered":"2024-05-08T22:35:09.391234+0000","last_unstale":"2024-05-08T22:35:42.328576+0000","last_undegraded":"2024-05-08T22:35:42.328576+0000","last_fullsized":"2024-05-08T22:35:42.328576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:37.226688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825589+0000","last_change":"2024-05-08T22:35:09.831973+0000","last_active":"2024-05-08T22:35:11.825589+0000","last_peered":"2024-05-08T22:35:11.825589+0000","last_clean":"2024-05-08T22:35:11.825589+0000","last_became_active":"2024-05-08T22:35:09.829817+0000","last_became_peered":"2024-05-08T22:35:09.829817+0000","last_unstale":"2024-05-08T22:35:11.825589+0000","last_undegraded":"2024-05-08T22:35:11.825589+0000","last_fullsized":"2024-05-08T22:35:11.825589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:53.829103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841201+0000","last_change":"2024-05-08T22:35:09.393821+0000","last_active":"2024-05-08T22:35:11.841201+0000","last_peered":"2024-05-08T22:35:11.841201+0000","last_clean":"2024-05-08T22:35:11.841201+0000","last_became_active":"2024-05-08T22:35:09.393503+0000","last_became_peered":"2024-05-08T22:35:09.393503+0000","last_unstale":"2024-05-08T22:35:11.841201+0000","last_undegraded":"2024-05-08T22:35:11.841201+0000","last_fullsized":"2024-05-08T22:35:11.841201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:56.252015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841407+0000","last_change":"2024-05-08T22:35:09.823341+0000","last_active":"2024-05-08T22:35:11.841407+0000","last_peered":"2024-05-08T22:35:11.841407+0000","last_clean":"2024-05-08T22:35:11.841407+0000","last_became_active":"2024-05-08T22:35:09.822949+0000","last_became_peered":"2024-05-08T22:35:09.822949+0000","last_unstale":"2024-05-08T22:35:11.841407+0000","last_undegraded":"2024-05-08T22:35:11.841407+0000","last_fullsized":"2024-05-08T22:35:11.841407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:49.189563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.268266+0000","last_change":"2024-05-08T22:35:09.391129+0000","last_active":"2024-05-08T22:35:15.268266+0000","last_peered":"2024-05-08T22:35:15.268266+0000","last_clean":"2024-05-08T22:35:15.268266+0000","last_became_active":"2024-05-08T22:35:09.390429+0000","last_became_peered":"2024-05-08T22:35:09.390429+0000","last_unstale":"2024-05-08T22:35:15.268266+0000","last_undegraded":"2024-05-08T22:35:15.268266+0000","last_fullsized":"2024-05-08T22:35:15.268266+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:18.416456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812741+0000","last_change":"2024-05-08T22:35:09.838274+0000","last_active":"2024-05-08T22:35:10.812741+0000","last_peered":"2024-05-08T22:35:10.812741+0000","last_clean":"2024-05-08T22:35:10.812741+0000","last_became_active":"2024-05-08T22:35:09.837637+0000","last_became_peered":"2024-05-08T22:35:09.837637+0000","last_unstale":"2024-05-08T22:35:10.812741+0000","last_undegraded":"2024-05-08T22:35:10.812741+0000","last_fullsized":"2024-05-08T22:35:10.812741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:00.476914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267817+0000","last_change":"2024-05-08T22:35:09.391987+0000","last_active":"2024-05-08T22:35:15.267817+0000","last_peered":"2024-05-08T22:35:15.267817+0000","last_clean":"2024-05-08T22:35:15.267817+0000","last_became_active":"2024-05-08T22:35:09.390587+0000","last_became_peered":"2024-05-08T22:35:09.390587+0000","last_unstale":"2024-05-08T22:35:15.267817+0000","last_undegraded":"2024-05-08T22:35:15.267817+0000","last_fullsized":"2024-05-08T22:35:15.267817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:14.343341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813185+0000","last_change":"2024-05-08T22:35:09.823177+0000","last_active":"2024-05-08T22:35:10.813185+0000","last_peered":"2024-05-08T22:35:10.813185+0000","last_clean":"2024-05-08T22:35:10.813185+0000","last_became_active":"2024-05-08T22:35:09.822830+0000","last_became_peered":"2024-05-08T22:35:09.822830+0000","last_unstale":"2024-05-08T22:35:10.813185+0000","last_undegraded":"2024-05-08T22:35:10.813185+0000","last_fullsized":"2024-05-08T22:35:10.813185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:44.648818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":48,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:40.301560+0000","last_change":"2024-05-08T22:35:09.392558+0000","last_active":"2024-05-08T22:35:40.301560+0000","last_peered":"2024-05-08T22:35:40.301560+0000","last_clean":"2024-05-08T22:35:40.301560+0000","last_became_active":"2024-05-08T22:35:09.392235+0000","last_became_peered":"2024-05-08T22:35:09.392235+0000","last_unstale":"2024-05-08T22:35:40.301560+0000","last_undegraded":"2024-05-08T22:35:40.301560+0000","last_fullsized":"2024-05-08T22:35:40.301560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:47:12.186400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.830402+0000","last_change":"2024-05-08T22:35:09.823455+0000","last_active":"2024-05-08T22:35:09.830402+0000","last_peered":"2024-05-08T22:35:09.830402+0000","last_clean":"2024-05-08T22:35:09.830402+0000","last_became_active":"2024-05-08T22:35:09.822818+0000","last_became_peered":"2024-05-08T22:35:09.822818+0000","last_unstale":"2024-05-08T22:35:09.830402+0000","last_undegraded":"2024-05-08T22:35:09.830402+0000","last_fullsized":"2024-05-08T22:35:09.830402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:38.142265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:14.857485+0000","last_change":"2024-05-08T22:35:09.394713+0000","last_active":"2024-05-08T22:35:14.857485+0000","last_peered":"2024-05-08T22:35:14.857485+0000","last_clean":"2024-05-08T22:35:14.857485+0000","last_became_active":"2024-05-08T22:35:09.394397+0000","last_became_peered":"2024-05-08T22:35:09.394397+0000","last_unstale":"2024-05-08T22:35:14.857485+0000","last_undegraded":"2024-05-08T22:35:14.857485+0000","last_fullsized":"2024-05-08T22:35:14.857485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:58:30.594207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813065+0000","last_change":"2024-05-08T22:35:09.827921+0000","last_active":"2024-05-08T22:35:10.813065+0000","last_peered":"2024-05-08T22:35:10.813065+0000","last_clean":"2024-05-08T22:35:10.813065+0000","last_became_active":"2024-05-08T22:35:09.827387+0000","last_became_peered":"2024-05-08T22:35:09.827387+0000","last_unstale":"2024-05-08T22:35:10.813065+0000","last_undegraded":"2024-05-08T22:35:10.813065+0000","last_fullsized":"2024-05-08T22:35:10.813065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:33.986935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.845741+0000","last_change":"2024-05-08T22:35:09.384387+0000","last_active":"2024-05-08T22:35:13.845741+0000","last_peered":"2024-05-08T22:35:13.845741+0000","last_clean":"2024-05-08T22:35:13.845741+0000","last_became_active":"2024-05-08T22:35:09.383970+0000","last_became_peered":"2024-05-08T22:35:09.383970+0000","last_unstale":"2024-05-08T22:35:13.845741+0000","last_undegraded":"2024-05-08T22:35:13.845741+0000","last_fullsized":"2024-05-08T22:35:13.845741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:55.430960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825444+0000","last_change":"2024-05-08T22:35:09.831477+0000","last_active":"2024-05-08T22:35:11.825444+0000","last_peered":"2024-05-08T22:35:11.825444+0000","last_clean":"2024-05-08T22:35:11.825444+0000","last_became_active":"2024-05-08T22:35:09.831168+0000","last_became_peered":"2024-05-08T22:35:09.831168+0000","last_unstale":"2024-05-08T22:35:11.825444+0000","last_undegraded":"2024-05-08T22:35:11.825444+0000","last_fullsized":"2024-05-08T22:35:11.825444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:56.765210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267651+0000","last_change":"2024-05-08T22:35:09.389063+0000","last_active":"2024-05-08T22:35:15.267651+0000","last_peered":"2024-05-08T22:35:15.267651+0000","last_clean":"2024-05-08T22:35:15.267651+0000","last_became_active":"2024-05-08T22:35:09.388683+0000","last_became_peered":"2024-05-08T22:35:09.388683+0000","last_unstale":"2024-05-08T22:35:15.267651+0000","last_undegraded":"2024-05-08T22:35:15.267651+0000","last_fullsized":"2024-05-08T22:35:15.267651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:03.804688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.842741+0000","last_change":"2024-05-08T22:35:09.829277+0000","last_active":"2024-05-08T22:35:11.842741+0000","last_peered":"2024-05-08T22:35:11.842741+0000","last_clean":"2024-05-08T22:35:11.842741+0000","last_became_active":"2024-05-08T22:35:09.828782+0000","last_became_peered":"2024-05-08T22:35:09.828782+0000","last_unstale":"2024-05-08T22:35:11.842741+0000","last_undegraded":"2024-05-08T22:35:11.842741+0000","last_fullsized":"2024-05-08T22:35:11.842741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:47.012528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":70,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:41.302457+0000","last_change":"2024-05-08T22:35:09.387464+0000","last_active":"2024-05-08T22:35:41.302457+0000","last_peered":"2024-05-08T22:35:41.302457+0000","last_clean":"2024-05-08T22:35:41.302457+0000","last_became_active":"2024-05-08T22:35:09.386994+0000","last_became_peered":"2024-05-08T22:35:09.386994+0000","last_unstale":"2024-05-08T22:35:41.302457+0000","last_undegraded":"2024-05-08T22:35:41.302457+0000","last_fullsized":"2024-05-08T22:35:41.302457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:27:04.151479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812918+0000","last_change":"2024-05-08T22:35:09.830886+0000","last_active":"2024-05-08T22:35:10.812918+0000","last_peered":"2024-05-08T22:35:10.812918+0000","last_clean":"2024-05-08T22:35:10.812918+0000","last_became_active":"2024-05-08T22:35:09.830466+0000","last_became_peered":"2024-05-08T22:35:09.830466+0000","last_unstale":"2024-05-08T22:35:10.812918+0000","last_undegraded":"2024-05-08T22:35:10.812918+0000","last_fullsized":"2024-05-08T22:35:10.812918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:12.337934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267954+0000","last_change":"2024-05-08T22:35:09.391510+0000","last_active":"2024-05-08T22:35:15.267954+0000","last_peered":"2024-05-08T22:35:15.267954+0000","last_clean":"2024-05-08T22:35:15.267954+0000","last_became_active":"2024-05-08T22:35:09.391145+0000","last_became_peered":"2024-05-08T22:35:09.391145+0000","last_unstale":"2024-05-08T22:35:15.267954+0000","last_undegraded":"2024-05-08T22:35:15.267954+0000","last_fullsized":"2024-05-08T22:35:15.267954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:39.694122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841304+0000","last_change":"2024-05-08T22:35:09.837056+0000","last_active":"2024-05-08T22:35:11.841304+0000","last_peered":"2024-05-08T22:35:11.841304+0000","last_clean":"2024-05-08T22:35:11.841304+0000","last_became_active":"2024-05-08T22:35:09.836724+0000","last_became_peered":"2024-05-08T22:35:09.836724+0000","last_unstale":"2024-05-08T22:35:11.841304+0000","last_undegraded":"2024-05-08T22:35:11.841304+0000","last_fullsized":"2024-05-08T22:35:11.841304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:48.901289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:14.855791+0000","last_change":"2024-05-08T22:35:09.395826+0000","last_active":"2024-05-08T22:35:14.855791+0000","last_peered":"2024-05-08T22:35:14.855791+0000","last_clean":"2024-05-08T22:35:14.855791+0000","last_became_active":"2024-05-08T22:35:09.395330+0000","last_became_peered":"2024-05-08T22:35:09.395330+0000","last_unstale":"2024-05-08T22:35:14.855791+0000","last_undegraded":"2024-05-08T22:35:14.855791+0000","last_fullsized":"2024-05-08T22:35:14.855791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:43.577894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812839+0000","last_change":"2024-05-08T22:35:09.831625+0000","last_active":"2024-05-08T22:35:10.812839+0000","last_peered":"2024-05-08T22:35:10.812839+0000","last_clean":"2024-05-08T22:35:10.812839+0000","last_became_active":"2024-05-08T22:35:09.831310+0000","last_became_peered":"2024-05-08T22:35:09.831310+0000","last_unstale":"2024-05-08T22:35:10.812839+0000","last_undegraded":"2024-05-08T22:35:10.812839+0000","last_fullsized":"2024-05-08T22:35:10.812839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:16.743871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'29","reported_seq":102,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:41.310964+0000","last_change":"2024-05-08T22:35:09.389805+0000","last_active":"2024-05-08T22:35:41.310964+0000","last_peered":"2024-05-08T22:35:41.310964+0000","last_clean":"2024-05-08T22:35:41.310964+0000","last_became_active":"2024-05-08T22:35:09.389443+0000","last_became_peered":"2024-05-08T22:35:09.389443+0000","last_unstale":"2024-05-08T22:35:41.310964+0000","last_undegraded":"2024-05-08T22:35:41.310964+0000","last_fullsized":"2024-05-08T22:35:41.310964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:40.479503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59419,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":88,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.835300+0000","last_change":"2024-05-08T22:35:09.834871+0000","last_active":"2024-05-08T22:35:09.835300+0000","last_peered":"2024-05-08T22:35:09.835300+0000","last_clean":"2024-05-08T22:35:09.835300+0000","last_became_active":"2024-05-08T22:35:09.834589+0000","last_became_peered":"2024-05-08T22:35:09.834589+0000","last_unstale":"2024-05-08T22:35:09.835300+0000","last_undegraded":"2024-05-08T22:35:09.835300+0000","last_fullsized":"2024-05-08T22:35:09.835300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:12.476142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267144+0000","last_change":"2024-05-08T22:35:09.388615+0000","last_active":"2024-05-08T22:35:15.267144+0000","last_peered":"2024-05-08T22:35:15.267144+0000","last_clean":"2024-05-08T22:35:15.267144+0000","last_became_active":"2024-05-08T22:35:09.388272+0000","last_became_peered":"2024-05-08T22:35:09.388272+0000","last_unstale":"2024-05-08T22:35:15.267144+0000","last_undegraded":"2024-05-08T22:35:15.267144+0000","last_fullsized":"2024-05-08T22:35:15.267144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:23.565450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470024+0000","last_change":"2024-05-08T22:35:09.831511+0000","last_active":"2024-05-08T22:35:10.470024+0000","last_peered":"2024-05-08T22:35:10.470024+0000","last_clean":"2024-05-08T22:35:10.470024+0000","last_became_active":"2024-05-08T22:35:09.831216+0000","last_became_peered":"2024-05-08T22:35:09.831216+0000","last_unstale":"2024-05-08T22:35:10.470024+0000","last_undegraded":"2024-05-08T22:35:10.470024+0000","last_fullsized":"2024-05-08T22:35:10.470024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:46.244544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.268190+0000","last_change":"2024-05-08T22:35:09.391835+0000","last_active":"2024-05-08T22:35:15.268190+0000","last_peered":"2024-05-08T22:35:15.268190+0000","last_clean":"2024-05-08T22:35:15.268190+0000","last_became_active":"2024-05-08T22:35:09.391465+0000","last_became_peered":"2024-05-08T22:35:09.391465+0000","last_unstale":"2024-05-08T22:35:15.268190+0000","last_undegraded":"2024-05-08T22:35:15.268190+0000","last_fullsized":"2024-05-08T22:35:15.268190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:30.819323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623032+0000","last_change":"2024-05-08T22:35:09.840351+0000","last_active":"2024-05-08T22:35:11.623032+0000","last_peered":"2024-05-08T22:35:11.623032+0000","last_clean":"2024-05-08T22:35:11.623032+0000","last_became_active":"2024-05-08T22:35:09.840000+0000","last_became_peered":"2024-05-08T22:35:09.840000+0000","last_unstale":"2024-05-08T22:35:11.623032+0000","last_undegraded":"2024-05-08T22:35:11.623032+0000","last_fullsized":"2024-05-08T22:35:11.623032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:51.428513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":51,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:43.313818+0000","last_change":"2024-05-08T22:35:09.389572+0000","last_active":"2024-05-08T22:35:43.313818+0000","last_peered":"2024-05-08T22:35:43.313818+0000","last_clean":"2024-05-08T22:35:43.313818+0000","last_became_active":"2024-05-08T22:35:09.389245+0000","last_became_peered":"2024-05-08T22:35:09.389245+0000","last_unstale":"2024-05-08T22:35:43.313818+0000","last_undegraded":"2024-05-08T22:35:43.313818+0000","last_fullsized":"2024-05-08T22:35:43.313818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:04.992010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.622950+0000","last_change":"2024-05-08T22:35:09.840132+0000","last_active":"2024-05-08T22:35:11.622950+0000","last_peered":"2024-05-08T22:35:11.622950+0000","last_clean":"2024-05-08T22:35:11.622950+0000","last_became_active":"2024-05-08T22:35:09.839684+0000","last_became_peered":"2024-05-08T22:35:09.839684+0000","last_unstale":"2024-05-08T22:35:11.622950+0000","last_undegraded":"2024-05-08T22:35:11.622950+0000","last_fullsized":"2024-05-08T22:35:11.622950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:32.325406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.258306+0000","last_change":"2024-05-08T22:35:09.392220+0000","last_active":"2024-05-08T22:35:15.258306+0000","last_peered":"2024-05-08T22:35:15.258306+0000","last_clean":"2024-05-08T22:35:15.258306+0000","last_became_active":"2024-05-08T22:35:09.391889+0000","last_became_peered":"2024-05-08T22:35:09.391889+0000","last_unstale":"2024-05-08T22:35:15.258306+0000","last_undegraded":"2024-05-08T22:35:15.258306+0000","last_fullsized":"2024-05-08T22:35:15.258306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:01.417894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813015+0000","last_change":"2024-05-08T22:35:09.823395+0000","last_active":"2024-05-08T22:35:10.813015+0000","last_peered":"2024-05-08T22:35:10.813015+0000","last_clean":"2024-05-08T22:35:10.813015+0000","last_became_active":"2024-05-08T22:35:09.822484+0000","last_became_peered":"2024-05-08T22:35:09.822484+0000","last_unstale":"2024-05-08T22:35:10.813015+0000","last_undegraded":"2024-05-08T22:35:10.813015+0000","last_fullsized":"2024-05-08T22:35:10.813015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:35.330789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.257929+0000","last_change":"2024-05-08T22:35:09.379904+0000","last_active":"2024-05-08T22:35:15.257929+0000","last_peered":"2024-05-08T22:35:15.257929+0000","last_clean":"2024-05-08T22:35:15.257929+0000","last_became_active":"2024-05-08T22:35:09.379533+0000","last_became_peered":"2024-05-08T22:35:09.379533+0000","last_unstale":"2024-05-08T22:35:15.257929+0000","last_undegraded":"2024-05-08T22:35:15.257929+0000","last_fullsized":"2024-05-08T22:35:15.257929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:14.625069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.838589+0000","last_change":"2024-05-08T22:35:09.836472+0000","last_active":"2024-05-08T22:35:09.838589+0000","last_peered":"2024-05-08T22:35:09.838589+0000","last_clean":"2024-05-08T22:35:09.838589+0000","last_became_active":"2024-05-08T22:35:09.836110+0000","last_became_peered":"2024-05-08T22:35:09.836110+0000","last_unstale":"2024-05-08T22:35:09.838589+0000","last_undegraded":"2024-05-08T22:35:09.838589+0000","last_fullsized":"2024-05-08T22:35:09.838589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:48.979661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.842537+0000","last_change":"2024-05-08T22:35:09.391647+0000","last_active":"2024-05-08T22:35:11.842537+0000","last_peered":"2024-05-08T22:35:11.842537+0000","last_clean":"2024-05-08T22:35:11.842537+0000","last_became_active":"2024-05-08T22:35:09.391273+0000","last_became_peered":"2024-05-08T22:35:09.391273+0000","last_unstale":"2024-05-08T22:35:11.842537+0000","last_undegraded":"2024-05-08T22:35:11.842537+0000","last_fullsized":"2024-05-08T22:35:11.842537+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:48.001313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.752901+0000","last_change":"2024-05-08T22:35:09.828713+0000","last_active":"2024-05-08T22:35:11.752901+0000","last_peered":"2024-05-08T22:35:11.752901+0000","last_clean":"2024-05-08T22:35:11.752901+0000","last_became_active":"2024-05-08T22:35:09.828370+0000","last_became_peered":"2024-05-08T22:35:09.828370+0000","last_unstale":"2024-05-08T22:35:11.752901+0000","last_undegraded":"2024-05-08T22:35:11.752901+0000","last_fullsized":"2024-05-08T22:35:11.752901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:23.328029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":44,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:40.327018+0000","last_change":"2024-05-08T22:35:09.391658+0000","last_active":"2024-05-08T22:35:40.327018+0000","last_peered":"2024-05-08T22:35:40.327018+0000","last_clean":"2024-05-08T22:35:40.327018+0000","last_became_active":"2024-05-08T22:35:09.391326+0000","last_became_peered":"2024-05-08T22:35:09.391326+0000","last_unstale":"2024-05-08T22:35:40.327018+0000","last_undegraded":"2024-05-08T22:35:40.327018+0000","last_fullsized":"2024-05-08T22:35:40.327018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:08:13.933145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604425+0000","last_change":"2024-05-08T22:35:09.837816+0000","last_active":"2024-05-08T22:35:11.604425+0000","last_peered":"2024-05-08T22:35:11.604425+0000","last_clean":"2024-05-08T22:35:11.604425+0000","last_became_active":"2024-05-08T22:35:09.837471+0000","last_became_peered":"2024-05-08T22:35:09.837471+0000","last_unstale":"2024-05-08T22:35:11.604425+0000","last_undegraded":"2024-05-08T22:35:11.604425+0000","last_fullsized":"2024-05-08T22:35:11.604425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:04.564055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813006+0000","last_change":"2024-05-08T22:35:09.827092+0000","last_active":"2024-05-08T22:35:10.813006+0000","last_peered":"2024-05-08T22:35:10.813006+0000","last_clean":"2024-05-08T22:35:10.813006+0000","last_became_active":"2024-05-08T22:35:09.826238+0000","last_became_peered":"2024-05-08T22:35:09.826238+0000","last_unstale":"2024-05-08T22:35:10.813006+0000","last_undegraded":"2024-05-08T22:35:10.813006+0000","last_fullsized":"2024-05-08T22:35:10.813006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:26:59.504555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":49,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:44.314446+0000","last_change":"2024-05-08T22:35:09.391555+0000","last_active":"2024-05-08T22:35:44.314446+0000","last_peered":"2024-05-08T22:35:44.314446+0000","last_clean":"2024-05-08T22:35:44.314446+0000","last_became_active":"2024-05-08T22:35:09.391294+0000","last_became_peered":"2024-05-08T22:35:09.391294+0000","last_unstale":"2024-05-08T22:35:44.314446+0000","last_undegraded":"2024-05-08T22:35:44.314446+0000","last_fullsized":"2024-05-08T22:35:44.314446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:41.195882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825544+0000","last_change":"2024-05-08T22:35:09.822814+0000","last_active":"2024-05-08T22:35:11.825544+0000","last_peered":"2024-05-08T22:35:11.825544+0000","last_clean":"2024-05-08T22:35:11.825544+0000","last_became_active":"2024-05-08T22:35:09.822425+0000","last_became_peered":"2024-05-08T22:35:09.822425+0000","last_unstale":"2024-05-08T22:35:11.825544+0000","last_undegraded":"2024-05-08T22:35:11.825544+0000","last_fullsized":"2024-05-08T22:35:11.825544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:56.212615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.265500+0000","last_change":"2024-05-08T22:35:09.383782+0000","last_active":"2024-05-08T22:35:15.265500+0000","last_peered":"2024-05-08T22:35:15.265500+0000","last_clean":"2024-05-08T22:35:15.265500+0000","last_became_active":"2024-05-08T22:35:09.383285+0000","last_became_peered":"2024-05-08T22:35:09.383285+0000","last_unstale":"2024-05-08T22:35:15.265500+0000","last_undegraded":"2024-05-08T22:35:15.265500+0000","last_fullsized":"2024-05-08T22:35:15.265500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:42.380683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":323,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841287+0000","last_change":"2024-05-08T22:34:20.037406+0000","last_active":"2024-05-08T22:35:11.841287+0000","last_peered":"2024-05-08T22:35:11.841287+0000","last_clean":"2024-05-08T22:35:11.841287+0000","last_became_active":"2024-05-08T22:34:20.036880+0000","last_became_peered":"2024-05-08T22:34:20.036880+0000","last_unstale":"2024-05-08T22:35:11.841287+0000","last_undegraded":"2024-05-08T22:35:11.841287+0000","last_fullsized":"2024-05-08T22:35:11.841287+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:31:54.953551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:31:54.953551+0000","last_clean_scrub_stamp":"2024-05-08T22:31:54.953551+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:00.820169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.469956+0000","last_change":"2024-05-08T22:35:09.830425+0000","last_active":"2024-05-08T22:35:10.469956+0000","last_peered":"2024-05-08T22:35:10.469956+0000","last_clean":"2024-05-08T22:35:10.469956+0000","last_became_active":"2024-05-08T22:35:09.830097+0000","last_became_peered":"2024-05-08T22:35:09.830097+0000","last_unstale":"2024-05-08T22:35:10.469956+0000","last_undegraded":"2024-05-08T22:35:10.469956+0000","last_fullsized":"2024-05-08T22:35:10.469956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:53.445798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.846273+0000","last_change":"2024-05-08T22:35:09.386717+0000","last_active":"2024-05-08T22:35:13.846273+0000","last_peered":"2024-05-08T22:35:13.846273+0000","last_clean":"2024-05-08T22:35:13.846273+0000","last_became_active":"2024-05-08T22:35:09.386371+0000","last_became_peered":"2024-05-08T22:35:09.386371+0000","last_unstale":"2024-05-08T22:35:13.846273+0000","last_undegraded":"2024-05-08T22:35:13.846273+0000","last_fullsized":"2024-05-08T22:35:13.846273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:41.061997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.257951+0000","last_change":"2024-05-08T22:35:09.394542+0000","last_active":"2024-05-08T22:35:15.257951+0000","last_peered":"2024-05-08T22:35:15.257951+0000","last_clean":"2024-05-08T22:35:15.257951+0000","last_became_active":"2024-05-08T22:35:09.394223+0000","last_became_peered":"2024-05-08T22:35:09.394223+0000","last_unstale":"2024-05-08T22:35:15.257951+0000","last_undegraded":"2024-05-08T22:35:15.257951+0000","last_fullsized":"2024-05-08T22:35:15.257951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:02.162377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470081+0000","last_change":"2024-05-08T22:35:09.825810+0000","last_active":"2024-05-08T22:35:10.470081+0000","last_peered":"2024-05-08T22:35:10.470081+0000","last_clean":"2024-05-08T22:35:10.470081+0000","last_became_active":"2024-05-08T22:35:09.825516+0000","last_became_peered":"2024-05-08T22:35:09.825516+0000","last_unstale":"2024-05-08T22:35:10.470081+0000","last_undegraded":"2024-05-08T22:35:10.470081+0000","last_fullsized":"2024-05-08T22:35:10.470081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:24.087187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'9","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:19.069957+0000","last_change":"2024-05-08T22:35:09.381688+0000","last_active":"2024-05-08T22:35:19.069957+0000","last_peered":"2024-05-08T22:35:19.069957+0000","last_clean":"2024-05-08T22:35:19.069957+0000","last_became_active":"2024-05-08T22:35:09.381343+0000","last_became_peered":"2024-05-08T22:35:09.381343+0000","last_unstale":"2024-05-08T22:35:19.069957+0000","last_undegraded":"2024-05-08T22:35:19.069957+0000","last_fullsized":"2024-05-08T22:35:19.069957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:14:16.142265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825676+0000","last_change":"2024-05-08T22:35:09.831689+0000","last_active":"2024-05-08T22:35:11.825676+0000","last_peered":"2024-05-08T22:35:11.825676+0000","last_clean":"2024-05-08T22:35:11.825676+0000","last_became_active":"2024-05-08T22:35:09.831337+0000","last_became_peered":"2024-05-08T22:35:09.831337+0000","last_unstale":"2024-05-08T22:35:11.825676+0000","last_undegraded":"2024-05-08T22:35:11.825676+0000","last_fullsized":"2024-05-08T22:35:11.825676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.965340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267698+0000","last_change":"2024-05-08T22:35:09.390479+0000","last_active":"2024-05-08T22:35:15.267698+0000","last_peered":"2024-05-08T22:35:15.267698+0000","last_clean":"2024-05-08T22:35:15.267698+0000","last_became_active":"2024-05-08T22:35:09.389790+0000","last_became_peered":"2024-05-08T22:35:09.389790+0000","last_unstale":"2024-05-08T22:35:15.267698+0000","last_undegraded":"2024-05-08T22:35:15.267698+0000","last_fullsized":"2024-05-08T22:35:15.267698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:27.668801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841685+0000","last_change":"2024-05-08T22:35:09.836123+0000","last_active":"2024-05-08T22:35:11.841685+0000","last_peered":"2024-05-08T22:35:11.841685+0000","last_clean":"2024-05-08T22:35:11.841685+0000","last_became_active":"2024-05-08T22:35:09.835840+0000","last_became_peered":"2024-05-08T22:35:09.835840+0000","last_unstale":"2024-05-08T22:35:11.841685+0000","last_undegraded":"2024-05-08T22:35:11.841685+0000","last_fullsized":"2024-05-08T22:35:11.841685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:52.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.840918+0000","last_change":"2024-05-08T22:35:09.394658+0000","last_active":"2024-05-08T22:35:12.840918+0000","last_peered":"2024-05-08T22:35:12.840918+0000","last_clean":"2024-05-08T22:35:12.840918+0000","last_became_active":"2024-05-08T22:35:09.394351+0000","last_became_peered":"2024-05-08T22:35:09.394351+0000","last_unstale":"2024-05-08T22:35:12.840918+0000","last_undegraded":"2024-05-08T22:35:12.840918+0000","last_fullsized":"2024-05-08T22:35:12.840918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:32.954612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.834139+0000","last_change":"2024-05-08T22:35:09.830853+0000","last_active":"2024-05-08T22:35:09.834139+0000","last_peered":"2024-05-08T22:35:09.834139+0000","last_clean":"2024-05-08T22:35:09.834139+0000","last_became_active":"2024-05-08T22:35:09.830471+0000","last_became_peered":"2024-05-08T22:35:09.830471+0000","last_unstale":"2024-05-08T22:35:09.834139+0000","last_undegraded":"2024-05-08T22:35:09.834139+0000","last_fullsized":"2024-05-08T22:35:09.834139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:13.960966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.265416+0000","last_change":"2024-05-08T22:35:09.382515+0000","last_active":"2024-05-08T22:35:15.265416+0000","last_peered":"2024-05-08T22:35:15.265416+0000","last_clean":"2024-05-08T22:35:15.265416+0000","last_became_active":"2024-05-08T22:35:09.382168+0000","last_became_peered":"2024-05-08T22:35:09.382168+0000","last_unstale":"2024-05-08T22:35:15.265416+0000","last_undegraded":"2024-05-08T22:35:15.265416+0000","last_fullsized":"2024-05-08T22:35:15.265416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:10.248542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.837958+0000","last_change":"2024-05-08T22:35:09.835925+0000","last_active":"2024-05-08T22:35:09.837958+0000","last_peered":"2024-05-08T22:35:09.837958+0000","last_clean":"2024-05-08T22:35:09.837958+0000","last_became_active":"2024-05-08T22:35:09.835547+0000","last_became_peered":"2024-05-08T22:35:09.835547+0000","last_unstale":"2024-05-08T22:35:09.837958+0000","last_undegraded":"2024-05-08T22:35:09.837958+0000","last_fullsized":"2024-05-08T22:35:09.837958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:36.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:14.857829+0000","last_change":"2024-05-08T22:35:09.385035+0000","last_active":"2024-05-08T22:35:14.857829+0000","last_peered":"2024-05-08T22:35:14.857829+0000","last_clean":"2024-05-08T22:35:14.857829+0000","last_became_active":"2024-05-08T22:35:09.384274+0000","last_became_peered":"2024-05-08T22:35:09.384274+0000","last_unstale":"2024-05-08T22:35:14.857829+0000","last_undegraded":"2024-05-08T22:35:14.857829+0000","last_fullsized":"2024-05-08T22:35:14.857829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:50.654985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825567+0000","last_change":"2024-05-08T22:35:09.821454+0000","last_active":"2024-05-08T22:35:11.825567+0000","last_peered":"2024-05-08T22:35:11.825567+0000","last_clean":"2024-05-08T22:35:11.825567+0000","last_became_active":"2024-05-08T22:35:09.821199+0000","last_became_peered":"2024-05-08T22:35:09.821199+0000","last_unstale":"2024-05-08T22:35:11.825567+0000","last_undegraded":"2024-05-08T22:35:11.825567+0000","last_fullsized":"2024-05-08T22:35:11.825567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:02.677307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.840831+0000","last_change":"2024-05-08T22:35:09.394027+0000","last_active":"2024-05-08T22:35:12.840831+0000","last_peered":"2024-05-08T22:35:12.840831+0000","last_clean":"2024-05-08T22:35:12.840831+0000","last_became_active":"2024-05-08T22:35:09.393622+0000","last_became_peered":"2024-05-08T22:35:09.393622+0000","last_unstale":"2024-05-08T22:35:12.840831+0000","last_undegraded":"2024-05-08T22:35:12.840831+0000","last_fullsized":"2024-05-08T22:35:12.840831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:13.415635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623012+0000","last_change":"2024-05-08T22:35:09.842928+0000","last_active":"2024-05-08T22:35:11.623012+0000","last_peered":"2024-05-08T22:35:11.623012+0000","last_clean":"2024-05-08T22:35:11.623012+0000","last_became_active":"2024-05-08T22:35:09.842576+0000","last_became_peered":"2024-05-08T22:35:09.842576+0000","last_unstale":"2024-05-08T22:35:11.623012+0000","last_undegraded":"2024-05-08T22:35:11.623012+0000","last_fullsized":"2024-05-08T22:35:11.623012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:03.526259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":48,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:42.304497+0000","last_change":"2024-05-08T22:35:09.396325+0000","last_active":"2024-05-08T22:35:42.304497+0000","last_peered":"2024-05-08T22:35:42.304497+0000","last_clean":"2024-05-08T22:35:42.304497+0000","last_became_active":"2024-05-08T22:35:09.395656+0000","last_became_peered":"2024-05-08T22:35:09.395656+0000","last_unstale":"2024-05-08T22:35:42.304497+0000","last_undegraded":"2024-05-08T22:35:42.304497+0000","last_fullsized":"2024-05-08T22:35:42.304497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:30.326694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623076+0000","last_change":"2024-05-08T22:35:09.840682+0000","last_active":"2024-05-08T22:35:11.623076+0000","last_peered":"2024-05-08T22:35:11.623076+0000","last_clean":"2024-05-08T22:35:11.623076+0000","last_became_active":"2024-05-08T22:35:09.840315+0000","last_became_peered":"2024-05-08T22:35:09.840315+0000","last_unstale":"2024-05-08T22:35:11.623076+0000","last_undegraded":"2024-05-08T22:35:11.623076+0000","last_fullsized":"2024-05-08T22:35:11.623076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:13.884569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.257932+0000","last_change":"2024-05-08T22:35:09.389169+0000","last_active":"2024-05-08T22:35:15.257932+0000","last_peered":"2024-05-08T22:35:15.257932+0000","last_clean":"2024-05-08T22:35:15.257932+0000","last_became_active":"2024-05-08T22:35:09.388836+0000","last_became_peered":"2024-05-08T22:35:09.388836+0000","last_unstale":"2024-05-08T22:35:15.257932+0000","last_undegraded":"2024-05-08T22:35:15.257932+0000","last_fullsized":"2024-05-08T22:35:15.257932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:08.960719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841499+0000","last_change":"2024-05-08T22:35:09.833885+0000","last_active":"2024-05-08T22:35:11.841499+0000","last_peered":"2024-05-08T22:35:11.841499+0000","last_clean":"2024-05-08T22:35:11.841499+0000","last_became_active":"2024-05-08T22:35:09.833596+0000","last_became_peered":"2024-05-08T22:35:09.833596+0000","last_unstale":"2024-05-08T22:35:11.841499+0000","last_undegraded":"2024-05-08T22:35:11.841499+0000","last_fullsized":"2024-05-08T22:35:11.841499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:29.673837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267349+0000","last_change":"2024-05-08T22:35:09.389023+0000","last_active":"2024-05-08T22:35:15.267349+0000","last_peered":"2024-05-08T22:35:15.267349+0000","last_clean":"2024-05-08T22:35:15.267349+0000","last_became_active":"2024-05-08T22:35:09.388721+0000","last_became_peered":"2024-05-08T22:35:09.388721+0000","last_unstale":"2024-05-08T22:35:15.267349+0000","last_undegraded":"2024-05-08T22:35:15.267349+0000","last_fullsized":"2024-05-08T22:35:15.267349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:41.776592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623109+0000","last_change":"2024-05-08T22:35:09.839499+0000","last_active":"2024-05-08T22:35:11.623109+0000","last_peered":"2024-05-08T22:35:11.623109+0000","last_clean":"2024-05-08T22:35:11.623109+0000","last_became_active":"2024-05-08T22:35:09.839146+0000","last_became_peered":"2024-05-08T22:35:09.839146+0000","last_unstale":"2024-05-08T22:35:11.623109+0000","last_undegraded":"2024-05-08T22:35:11.623109+0000","last_fullsized":"2024-05-08T22:35:11.623109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:45.636827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.839015+0000","last_change":"2024-05-08T22:35:09.836710+0000","last_active":"2024-05-08T22:35:09.839015+0000","last_peered":"2024-05-08T22:35:09.839015+0000","last_clean":"2024-05-08T22:35:09.839015+0000","last_became_active":"2024-05-08T22:35:09.835838+0000","last_became_peered":"2024-05-08T22:35:09.835838+0000","last_unstale":"2024-05-08T22:35:09.839015+0000","last_undegraded":"2024-05-08T22:35:09.839015+0000","last_fullsized":"2024-05-08T22:35:09.839015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:00.567805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841269+0000","last_change":"2024-05-08T22:35:09.830148+0000","last_active":"2024-05-08T22:35:11.841269+0000","last_peered":"2024-05-08T22:35:11.841269+0000","last_clean":"2024-05-08T22:35:11.841269+0000","last_became_active":"2024-05-08T22:35:09.829820+0000","last_became_peered":"2024-05-08T22:35:09.829820+0000","last_unstale":"2024-05-08T22:35:11.841269+0000","last_undegraded":"2024-05-08T22:35:11.841269+0000","last_fullsized":"2024-05-08T22:35:11.841269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:38.840993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841227+0000","last_change":"2024-05-08T22:35:09.830428+0000","last_active":"2024-05-08T22:35:11.841227+0000","last_peered":"2024-05-08T22:35:11.841227+0000","last_clean":"2024-05-08T22:35:11.841227+0000","last_became_active":"2024-05-08T22:35:09.829736+0000","last_became_peered":"2024-05-08T22:35:09.829736+0000","last_unstale":"2024-05-08T22:35:11.841227+0000","last_undegraded":"2024-05-08T22:35:11.841227+0000","last_fullsized":"2024-05-08T22:35:11.841227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:54.458662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841649+0000","last_change":"2024-05-08T22:35:09.836200+0000","last_active":"2024-05-08T22:35:11.841649+0000","last_peered":"2024-05-08T22:35:11.841649+0000","last_clean":"2024-05-08T22:35:11.841649+0000","last_became_active":"2024-05-08T22:35:09.835892+0000","last_became_peered":"2024-05-08T22:35:09.835892+0000","last_unstale":"2024-05-08T22:35:11.841649+0000","last_undegraded":"2024-05-08T22:35:11.841649+0000","last_fullsized":"2024-05-08T22:35:11.841649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:42.487982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470102+0000","last_change":"2024-05-08T22:35:09.834026+0000","last_active":"2024-05-08T22:35:10.470102+0000","last_peered":"2024-05-08T22:35:10.470102+0000","last_clean":"2024-05-08T22:35:10.470102+0000","last_became_active":"2024-05-08T22:35:09.833659+0000","last_became_peered":"2024-05-08T22:35:09.833659+0000","last_unstale":"2024-05-08T22:35:10.470102+0000","last_undegraded":"2024-05-08T22:35:10.470102+0000","last_fullsized":"2024-05-08T22:35:10.470102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:00.536853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.839061+0000","last_change":"2024-05-08T22:35:09.836928+0000","last_active":"2024-05-08T22:35:09.839061+0000","last_peered":"2024-05-08T22:35:09.839061+0000","last_clean":"2024-05-08T22:35:09.839061+0000","last_became_active":"2024-05-08T22:35:09.836629+0000","last_became_peered":"2024-05-08T22:35:09.836629+0000","last_unstale":"2024-05-08T22:35:09.839061+0000","last_undegraded":"2024-05-08T22:35:09.839061+0000","last_fullsized":"2024-05-08T22:35:09.839061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:43.475121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841544+0000","last_change":"2024-05-08T22:35:09.837685+0000","last_active":"2024-05-08T22:35:11.841544+0000","last_peered":"2024-05-08T22:35:11.841544+0000","last_clean":"2024-05-08T22:35:11.841544+0000","last_became_active":"2024-05-08T22:35:09.837409+0000","last_became_peered":"2024-05-08T22:35:09.837409+0000","last_unstale":"2024-05-08T22:35:11.841544+0000","last_undegraded":"2024-05-08T22:35:11.841544+0000","last_fullsized":"2024-05-08T22:35:11.841544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:32.076361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825653+0000","last_change":"2024-05-08T22:35:09.833099+0000","last_active":"2024-05-08T22:35:11.825653+0000","last_peered":"2024-05-08T22:35:11.825653+0000","last_clean":"2024-05-08T22:35:11.825653+0000","last_became_active":"2024-05-08T22:35:09.832814+0000","last_became_peered":"2024-05-08T22:35:09.832814+0000","last_unstale":"2024-05-08T22:35:11.825653+0000","last_undegraded":"2024-05-08T22:35:11.825653+0000","last_fullsized":"2024-05-08T22:35:11.825653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:40.116060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812821+0000","last_change":"2024-05-08T22:35:09.818507+0000","last_active":"2024-05-08T22:35:10.812821+0000","last_peered":"2024-05-08T22:35:10.812821+0000","last_clean":"2024-05-08T22:35:10.812821+0000","last_became_active":"2024-05-08T22:35:09.818139+0000","last_became_peered":"2024-05-08T22:35:09.818139+0000","last_unstale":"2024-05-08T22:35:10.812821+0000","last_undegraded":"2024-05-08T22:35:10.812821+0000","last_fullsized":"2024-05-08T22:35:10.812821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:33.747011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825503+0000","last_change":"2024-05-08T22:35:09.832232+0000","last_active":"2024-05-08T22:35:11.825503+0000","last_peered":"2024-05-08T22:35:11.825503+0000","last_clean":"2024-05-08T22:35:11.825503+0000","last_became_active":"2024-05-08T22:35:09.831889+0000","last_became_peered":"2024-05-08T22:35:09.831889+0000","last_unstale":"2024-05-08T22:35:11.825503+0000","last_undegraded":"2024-05-08T22:35:11.825503+0000","last_fullsized":"2024-05-08T22:35:11.825503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:01.802009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812645+0000","last_change":"2024-05-08T22:35:09.838051+0000","last_active":"2024-05-08T22:35:10.812645+0000","last_peered":"2024-05-08T22:35:10.812645+0000","last_clean":"2024-05-08T22:35:10.812645+0000","last_became_active":"2024-05-08T22:35:09.837429+0000","last_became_peered":"2024-05-08T22:35:09.837429+0000","last_unstale":"2024-05-08T22:35:10.812645+0000","last_undegraded":"2024-05-08T22:35:10.812645+0000","last_fullsized":"2024-05-08T22:35:10.812645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.191374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470060+0000","last_change":"2024-05-08T22:35:09.830834+0000","last_active":"2024-05-08T22:35:10.470060+0000","last_peered":"2024-05-08T22:35:10.470060+0000","last_clean":"2024-05-08T22:35:10.470060+0000","last_became_active":"2024-05-08T22:35:09.830475+0000","last_became_peered":"2024-05-08T22:35:09.830475+0000","last_unstale":"2024-05-08T22:35:10.470060+0000","last_undegraded":"2024-05-08T22:35:10.470060+0000","last_fullsized":"2024-05-08T22:35:10.470060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:06:10.996668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.838544+0000","last_change":"2024-05-08T22:35:09.836496+0000","last_active":"2024-05-08T22:35:09.838544+0000","last_peered":"2024-05-08T22:35:09.838544+0000","last_clean":"2024-05-08T22:35:09.838544+0000","last_became_active":"2024-05-08T22:35:09.835787+0000","last_became_peered":"2024-05-08T22:35:09.835787+0000","last_unstale":"2024-05-08T22:35:09.838544+0000","last_undegraded":"2024-05-08T22:35:09.838544+0000","last_fullsized":"2024-05-08T22:35:09.838544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:30.395286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66435,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":318,"num_read_kb":233,"num_write":230,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":864256,"data_stored":410583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70599999999999996}]},{"osd":2,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.629},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.69499999999999995}]},{"osd":7,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78400000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":9,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.91800000000000004}]},{"osd":10,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.83099999999999996}]}]},{"osd":10,"up_from":67,"seq":287762808853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32636,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712516,"statfs":{"total":95995035648,"available":95961616384,"internally_reserved":0,"allocated":1560576,"data_stored":1061057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.70899999999999996}]},{"osd":3,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64400000000000002}]},{"osd":4,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61099999999999999}]},{"osd":5,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66700000000000004}]},{"osd":6,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.628},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.753}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.88100000000000001}]},{"osd":9,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.84899999999999998}]},{"osd":11,"last update":"Wed May 8 22:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.89500000000000002}]}]},{"osd":9,"up_from":62,"seq":266287972377,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":888832,"data_stored":410561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.314},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.40899999999999997}]},{"osd":3,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":1.893},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.34399999999999997}]},{"osd":4,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.55200000000000005}]},{"osd":5,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.754},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":1.8300000000000001}]},{"osd":7,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.65900000000000003}]},{"osd":8,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.63},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.798}]},{"osd":10,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.8620000000000001}]}]},{"osd":8,"up_from":55,"seq":236223201308,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":880640,"data_stored":410658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65800000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.51800000000000002}]},{"osd":1,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80300000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81100000000000005,"5min":1.014,"15min":1.014},"last":0.72999999999999998}]},{"osd":2,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.435,"15min":0.433},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65100000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.501,"5min":0.47299999999999998,"15min":0.46800000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85399999999999998,"15min":0.85299999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.4630000000000001,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.35099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85099999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83699999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.623}]},{"osd":5,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76300000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72499999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54600000000000004}]},{"osd":6,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.95499999999999996,"15min":1.0089999999999999},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92800000000000005,"5min":7.3689999999999998,"15min":7.3689999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.202,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.82899999999999996}]},{"osd":7,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88600000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.81499999999999995}]},{"osd":9,"last update":"Wed May 8 22:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.749},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.79900000000000004}]},{"osd":10,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.88300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]}]},{"osd":7,"up_from":48,"seq":206158430239,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32556,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712596,"statfs":{"total":95995035648,"available":95961698304,"internally_reserved":0,"allocated":1478656,"data_stored":1003377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40100000000000002}]},{"osd":1,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26500000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.46899999999999997},"min":{"1min":0.33200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.373,"5min":0.222,"15min":0.222},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.48199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.53200000000000003}]},{"osd":3,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.627,"15min":0.58699999999999997},"min":{"1min":0.39200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.4809999999999999,"5min":5.4809999999999999,"15min":5.4809999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71999999999999997}]},{"osd":4,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.35499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76900000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.69099999999999995}]},{"osd":5,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.66700000000000004,"15min":0.64600000000000002},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.7229999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.749},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.48799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":0.79300000000000004}]},{"osd":6,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.76700000000000002,"15min":0.71899999999999997},"min":{"1min":0.54200000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.5650000000000004,"5min":6.5650000000000004,"15min":6.5650000000000004},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.72399999999999998,"15min":0.70799999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.77700000000000002}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.86399999999999999}]},{"osd":9,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.88}]},{"osd":10,"last update":"Wed May 8 22:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.91900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":6,"up_from":43,"seq":184683593763,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":410230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39000000000000001,"15min":0.38800000000000001},"min":{"1min":0.29399999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.316},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42499999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.28999999999999998}]},{"osd":1,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.82399999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77000000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.44600000000000001}]},{"osd":3,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47699999999999998,"15min":0.48599999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56999999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.376},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.49299999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.35499999999999998}]},{"osd":4,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63},"min":{"1min":0.44800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77500000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.874,"15min":0.874},"last":0.499}]},{"osd":5,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.48199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.39500000000000002}]},{"osd":7,"last update":"Wed May 8 22:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92700000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.79700000000000004,"5min":0.874,"15min":0.874},"last":0.56699999999999995}]},{"osd":8,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94499999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71499999999999997,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.0149999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.48399999999999999}]},{"osd":9,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.52100000000000002}]},{"osd":10,"last update":"Wed May 8 22:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.69999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":929792,"data_stored":470289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48399999999999999,"15min":0.47199999999999998},"min":{"1min":0.317,"5min":0.311,"15min":0.311},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.58199999999999996}]},{"osd":2,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83599999999999997,"5min":1.627,"15min":1.627},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.91900000000000004}]},{"osd":3,"last update":"Wed May 8 22:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.35299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.40600000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.89300000000000002,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.70299999999999996}]},{"osd":4,"last update":"Wed May 8 22:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.44500000000000001,"15min":0.42499999999999999},"min":{"1min":0.33700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.47799999999999998,"15min":0.44400000000000001},"min":{"1min":0.33100000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.73799999999999999}]},{"osd":5,"last update":"Wed May 8 22:34:52 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.97699999999999998}]},{"osd":6,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.47599999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Wed May 8 22:35:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.872},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.90300000000000002}]},{"osd":8,"last update":"Wed May 8 22:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":1.0269999999999999}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.98699999999999999}]},{"osd":10,"last update":"Wed May 8 22:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":411569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.36399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.83699999999999997,"5min":0.996,"15min":0.996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.502},"min":{"1min":0.33500000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.74099999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed May 8 22:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.40200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51400000000000001,"15min":0.48799999999999999},"min":{"1min":0.36099999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.76500000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed May 8 22:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.39600000000000002,"15min":0.38700000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.42699999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 22:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56100000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.78700000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.53600000000000003,"15min":0.52900000000000003},"min":{"1min":0.38200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.69599999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48299999999999998}]},{"osd":5,"last update":"Wed May 8 22:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70599999999999996,"5min":0.751,"15min":0.751},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90000000000000002,"5min":1.085,"15min":1.085},"last":0.624}]},{"osd":6,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78700000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83899999999999997,"5min":0.872,"15min":0.872},"last":0.5}]},{"osd":7,"last update":"Wed May 8 22:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53500000000000003}]},{"osd":8,"last update":"Wed May 8 22:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.79900000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.745,"15min":0.76500000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.81200000000000006,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.63900000000000001}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.58899999999999997}]},{"osd":10,"last update":"Wed May 8 22:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.54000000000000004}]},{"osd":11,"last update":"Wed May 8 22:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65900000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":950272,"data_stored":470458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71799999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48599999999999999}]},{"osd":1,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.40500000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68200000000000005}]},{"osd":3,"last update":"Wed May 8 22:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69199999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.50800000000000001,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed May 8 22:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59599999999999997}]},{"osd":5,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.33200000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64100000000000001,"5min":0.748,"15min":0.748},"last":0.38700000000000001}]},{"osd":6,"last update":"Wed May 8 22:35:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.50600000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64400000000000002}]},{"osd":7,"last update":"Wed May 8 22:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.47299999999999998}]},{"osd":8,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65400000000000003}]},{"osd":9,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70399999999999996}]},{"osd":10,"last update":"Wed May 8 22:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":3,"up_from":24,"seq":103079215149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36076,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709076,"statfs":{"total":95995035648,"available":95958093824,"internally_reserved":0,"allocated":888832,"data_stored":410103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.33600000000000002},"min":{"1min":0.27200000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.54100000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.36399999999999999,"15min":0.35199999999999998},"min":{"1min":0.28899999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58899999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.315}]},{"osd":1,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72999999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54100000000000004,"15min":0.52800000000000002},"min":{"1min":0.38800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88100000000000001,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.46800000000000003}]},{"osd":4,"last update":"Wed May 8 22:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56200000000000006,"15min":0.53800000000000003},"min":{"1min":0.44800000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78600000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80600000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed May 8 22:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85499999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.52500000000000002}]},{"osd":6,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.38}]},{"osd":7,"last update":"Wed May 8 22:35:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.63200000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73299999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.47899999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91400000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69699999999999995}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.496},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.53900000000000003}]},{"osd":9,"last update":"Wed May 8 22:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.41999999999999998}]},{"osd":10,"last update":"Wed May 8 22:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.625},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.71799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":872448,"data_stored":410484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.38100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.42699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40100000000000002},"min":{"1min":0.29899999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64800000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.504},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.41599999999999998},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.64500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.40600000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58399999999999996,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66600000000000004}]},{"osd":3,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.34499999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66600000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53300000000000003,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.318,"15min":0.318},"max":{"1min":0.64700000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56000000000000005}]},{"osd":5,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89200000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.89000000000000001}]},{"osd":6,"last update":"Wed May 8 22:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.40699999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed May 8 22:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94599999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.75700000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61699999999999999}]},{"osd":8,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78700000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.93300000000000005}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.90500000000000003}]},{"osd":10,"last update":"Wed May 8 22:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.94299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":5,"up_from":36,"seq":154618822694,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1490944,"data_stored":1002758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.50800000000000001,"15min":0.499},"min":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.35899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.747},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.69599999999999995}]},{"osd":2,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.379,"15min":0.372},"min":{"1min":0.28199999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.443}]},{"osd":3,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.49099999999999999}]},{"osd":4,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.67500000000000004}]},{"osd":6,"last update":"Wed May 8 22:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40899999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":1.171}]},{"osd":7,"last update":"Wed May 8 22:35:27 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81499999999999995,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86899999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.60299999999999998}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.57899999999999996}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56699999999999995}]},{"osd":10,"last update":"Wed May 8 22:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:35:47.369 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T22:35:47.600 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:48.017 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:47 smithi062 ceph-mon[42571]: pgmap v240: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-08T22:35:48.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:47 smithi066 ceph-mon[46830]: pgmap v240: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-08T22:35:48.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:47 smithi079 ceph-mon[46738]: pgmap v240: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-08T22:35:48.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.555+0000 7fc979e7c700 1 -- 172.21.15.62:0/3449920426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc974105f00 msgr2=0x7fc9741062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:48.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.555+0000 7fc979e7c700 1 --2- 172.21.15.62:0/3449920426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc974105f00 0x7fc9741062e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc964009990 tx=0x7fc964009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.555+0000 7fc979e7c700 1 -- 172.21.15.62:0/3449920426 shutdown_connections 2024-05-08T22:35:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.555+0000 7fc979e7c700 1 --2- 172.21.15.62:0/3449920426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc974100890 0x7fc97410cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.555+0000 7fc979e7c700 1 --2- 172.21.15.62:0/3449920426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc974105f00 0x7fc9741062e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.555+0000 7fc979e7c700 1 --2- 172.21.15.62:0/3449920426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9740ffed0 0x7fc974100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.555+0000 7fc979e7c700 1 -- 172.21.15.62:0/3449920426 >> 172.21.15.62:0/3449920426 conn(0x7fc974075f80 msgr2=0x7fc974076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.555+0000 7fc979e7c700 1 -- 172.21.15.62:0/3449920426 shutdown_connections 2024-05-08T22:35:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.555+0000 7fc979e7c700 1 -- 172.21.15.62:0/3449920426 wait complete. 2024-05-08T22:35:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc979e7c700 1 Processor -- start 2024-05-08T22:35:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc979e7c700 1 -- start start 2024-05-08T22:35:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc979e7c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9740ffed0 0x7fc974101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:48.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc979e7c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc974100890 0x7fc9741024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc979e7c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc974105f00 0x7fc9741029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc979e7c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9741a5830 con 0x7fc974100890 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc979e7c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9741a59a0 con 0x7fc974105f00 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc979e7c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9741a5b60 con 0x7fc9740ffed0 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc972ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc974100890 0x7fc9741024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc972ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc974100890 0x7fc9741024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44024/0 (socket says 172.21.15.62:44024) 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc972ffd700 1 -- 172.21.15.62:0/1028436500 learned_addr learned my addr 172.21.15.62:0/1028436500 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.556+0000 7fc973fff700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc974105f00 0x7fc9741029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc9737fe700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9740ffed0 0x7fc974101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc972ffd700 1 -- 172.21.15.62:0/1028436500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9740ffed0 msgr2=0x7fc974101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc972ffd700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9740ffed0 0x7fc974101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc972ffd700 1 -- 172.21.15.62:0/1028436500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc974105f00 msgr2=0x7fc9741029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc972ffd700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc974105f00 0x7fc9741029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc972ffd700 1 -- 172.21.15.62:0/1028436500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9640095f0 con 0x7fc974100890 2024-05-08T22:35:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc973fff700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc974105f00 0x7fc9741029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc972ffd700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc974100890 0x7fc9741024a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc96800b530 tx=0x7fc96800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc970ff9700 1 -- 172.21.15.62:0/1028436500 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968011830 con 0x7fc974100890 2024-05-08T22:35:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc979e7c700 1 -- 172.21.15.62:0/1028436500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9741a5e40 con 0x7fc974100890 2024-05-08T22:35:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc970ff9700 1 -- 172.21.15.62:0/1028436500 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc968011e50 con 0x7fc974100890 2024-05-08T22:35:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc970ff9700 1 -- 172.21.15.62:0/1028436500 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96800f2e0 con 0x7fc974100890 2024-05-08T22:35:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.557+0000 7fc979e7c700 1 -- 172.21.15.62:0/1028436500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9741a62b0 con 0x7fc974100890 2024-05-08T22:35:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.558+0000 7fc979e7c700 1 -- 172.21.15.62:0/1028436500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9741015b0 con 0x7fc974100890 2024-05-08T22:35:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.559+0000 7fc970ff9700 1 -- 172.21.15.62:0/1028436500 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc968010580 con 0x7fc974100890 2024-05-08T22:35:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.560+0000 7fc970ff9700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc95c0a11b0 0x7fc95c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.560+0000 7fc9737fe700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc95c0a11b0 0x7fc95c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.560+0000 7fc970ff9700 1 -- 172.21.15.62:0/1028436500 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc9680cc2d0 con 0x7fc974100890 2024-05-08T22:35:48.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.560+0000 7fc9737fe700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc95c0a11b0 0x7fc95c0a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc964006010 tx=0x7fc96400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:48.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.565+0000 7fc970ff9700 1 -- 172.21.15.62:0/1028436500 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc968095bc0 con 0x7fc974100890 2024-05-08T22:35:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.762+0000 7fc979e7c700 1 -- 172.21.15.62:0/1028436500 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc97410feb0 con 0x7fc95c0a11b0 2024-05-08T22:35:48.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.772+0000 7fc970ff9700 1 -- 172.21.15.62:0/1028436500 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340893 (secure 0 0 0) 0x7fc97410feb0 con 0x7fc95c0a11b0 2024-05-08T22:35:48.777 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 -- 172.21.15.62:0/1028436500 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc95c0a11b0 msgr2=0x7fc95c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc95c0a11b0 0x7fc95c0a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc964006010 tx=0x7fc96400b540 comp rx=0 tx=0).stop 2024-05-08T22:35:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 -- 172.21.15.62:0/1028436500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc974100890 msgr2=0x7fc9741024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc974100890 0x7fc9741024a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc96800b530 tx=0x7fc96800b8f0 comp rx=0 tx=0).stop 2024-05-08T22:35:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 -- 172.21.15.62:0/1028436500 shutdown_connections 2024-05-08T22:35:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc95c0a11b0 0x7fc95c0a3670 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc974105f00 0x7fc9741029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9740ffed0 0x7fc974101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 --2- 172.21.15.62:0/1028436500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc974100890 0x7fc9741024a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 -- 172.21.15.62:0/1028436500 >> 172.21.15.62:0/1028436500 conn(0x7fc974075f80 msgr2=0x7fc974104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 -- 172.21.15.62:0/1028436500 shutdown_connections 2024-05-08T22:35:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:48.776+0000 7fc979e7c700 1 -- 172.21.15.62:0/1028436500 wait complete. 2024-05-08T22:35:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-05-08T22:35:49.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:48 smithi062 ceph-mon[42571]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:49.139 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-05-08T22:35:48.024089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657037,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":474,"num_read_kb":357,"num_write":413,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381524,"kb_used_data":12276,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124560300,"statfs":{"total":1151940427776,"available":1151549747200,"internally_reserved":0,"allocated":12570624,"data_stored":6882127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":377992518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1138,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":66,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003290"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.840865+0000","last_change":"2024-05-08T22:35:09.840453+0000","last_active":"2024-05-08T22:35:09.840865+0000","last_peered":"2024-05-08T22:35:09.840865+0000","last_clean":"2024-05-08T22:35:09.840865+0000","last_became_active":"2024-05-08T22:35:09.840093+0000","last_became_peered":"2024-05-08T22:35:09.840093+0000","last_unstale":"2024-05-08T22:35:09.840865+0000","last_undegraded":"2024-05-08T22:35:09.840865+0000","last_fullsized":"2024-05-08T22:35:09.840865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:13.265286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840996+0000","last_change":"2024-05-08T22:35:09.832735+0000","last_active":"2024-05-08T22:35:11.840996+0000","last_peered":"2024-05-08T22:35:11.840996+0000","last_clean":"2024-05-08T22:35:11.840996+0000","last_became_active":"2024-05-08T22:35:09.832419+0000","last_became_peered":"2024-05-08T22:35:09.832419+0000","last_unstale":"2024-05-08T22:35:11.840996+0000","last_undegraded":"2024-05-08T22:35:11.840996+0000","last_fullsized":"2024-05-08T22:35:11.840996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:19.711398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841216+0000","last_change":"2024-05-08T22:35:09.835604+0000","last_active":"2024-05-08T22:35:11.841216+0000","last_peered":"2024-05-08T22:35:11.841216+0000","last_clean":"2024-05-08T22:35:11.841216+0000","last_became_active":"2024-05-08T22:35:09.835235+0000","last_became_peered":"2024-05-08T22:35:09.835235+0000","last_unstale":"2024-05-08T22:35:11.841216+0000","last_undegraded":"2024-05-08T22:35:11.841216+0000","last_fullsized":"2024-05-08T22:35:11.841216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:35:10.722947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.831424+0000","last_change":"2024-05-08T22:35:09.824876+0000","last_active":"2024-05-08T22:35:09.831424+0000","last_peered":"2024-05-08T22:35:09.831424+0000","last_clean":"2024-05-08T22:35:09.831424+0000","last_became_active":"2024-05-08T22:35:09.823494+0000","last_became_peered":"2024-05-08T22:35:09.823494+0000","last_unstale":"2024-05-08T22:35:09.831424+0000","last_undegraded":"2024-05-08T22:35:09.831424+0000","last_fullsized":"2024-05-08T22:35:09.831424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:20.099947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.828965+0000","last_change":"2024-05-08T22:35:09.825647+0000","last_active":"2024-05-08T22:35:09.828965+0000","last_peered":"2024-05-08T22:35:09.828965+0000","last_clean":"2024-05-08T22:35:09.828965+0000","last_became_active":"2024-05-08T22:35:09.825322+0000","last_became_peered":"2024-05-08T22:35:09.825322+0000","last_unstale":"2024-05-08T22:35:09.828965+0000","last_undegraded":"2024-05-08T22:35:09.828965+0000","last_fullsized":"2024-05-08T22:35:09.828965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:29.080541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604425+0000","last_change":"2024-05-08T22:35:09.841894+0000","last_active":"2024-05-08T22:35:11.604425+0000","last_peered":"2024-05-08T22:35:11.604425+0000","last_clean":"2024-05-08T22:35:11.604425+0000","last_became_active":"2024-05-08T22:35:09.841429+0000","last_became_peered":"2024-05-08T22:35:09.841429+0000","last_unstale":"2024-05-08T22:35:11.604425+0000","last_undegraded":"2024-05-08T22:35:11.604425+0000","last_fullsized":"2024-05-08T22:35:11.604425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:50.859186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.622990+0000","last_change":"2024-05-08T22:35:09.834010+0000","last_active":"2024-05-08T22:35:11.622990+0000","last_peered":"2024-05-08T22:35:11.622990+0000","last_clean":"2024-05-08T22:35:11.622990+0000","last_became_active":"2024-05-08T22:35:09.833633+0000","last_became_peered":"2024-05-08T22:35:09.833633+0000","last_unstale":"2024-05-08T22:35:11.622990+0000","last_undegraded":"2024-05-08T22:35:11.622990+0000","last_fullsized":"2024-05-08T22:35:11.622990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:28.854383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825479+0000","last_change":"2024-05-08T22:35:09.820817+0000","last_active":"2024-05-08T22:35:11.825479+0000","last_peered":"2024-05-08T22:35:11.825479+0000","last_clean":"2024-05-08T22:35:11.825479+0000","last_became_active":"2024-05-08T22:35:09.820521+0000","last_became_peered":"2024-05-08T22:35:09.820521+0000","last_unstale":"2024-05-08T22:35:11.825479+0000","last_undegraded":"2024-05-08T22:35:11.825479+0000","last_fullsized":"2024-05-08T22:35:11.825479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:17.312794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812985+0000","last_change":"2024-05-08T22:35:09.838821+0000","last_active":"2024-05-08T22:35:10.812985+0000","last_peered":"2024-05-08T22:35:10.812985+0000","last_clean":"2024-05-08T22:35:10.812985+0000","last_became_active":"2024-05-08T22:35:09.836588+0000","last_became_peered":"2024-05-08T22:35:09.836588+0000","last_unstale":"2024-05-08T22:35:10.812985+0000","last_undegraded":"2024-05-08T22:35:10.812985+0000","last_fullsized":"2024-05-08T22:35:10.812985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:52.884827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604696+0000","last_change":"2024-05-08T22:35:09.836222+0000","last_active":"2024-05-08T22:35:11.604696+0000","last_peered":"2024-05-08T22:35:11.604696+0000","last_clean":"2024-05-08T22:35:11.604696+0000","last_became_active":"2024-05-08T22:35:09.835810+0000","last_became_peered":"2024-05-08T22:35:09.835810+0000","last_unstale":"2024-05-08T22:35:11.604696+0000","last_undegraded":"2024-05-08T22:35:11.604696+0000","last_fullsized":"2024-05-08T22:35:11.604696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:38:14.075576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623061+0000","last_change":"2024-05-08T22:35:09.835422+0000","last_active":"2024-05-08T22:35:11.623061+0000","last_peered":"2024-05-08T22:35:11.623061+0000","last_clean":"2024-05-08T22:35:11.623061+0000","last_became_active":"2024-05-08T22:35:09.835100+0000","last_became_peered":"2024-05-08T22:35:09.835100+0000","last_unstale":"2024-05-08T22:35:11.623061+0000","last_undegraded":"2024-05-08T22:35:11.623061+0000","last_fullsized":"2024-05-08T22:35:11.623061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:12.099286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.832873+0000","last_change":"2024-05-08T22:35:09.827007+0000","last_active":"2024-05-08T22:35:09.832873+0000","last_peered":"2024-05-08T22:35:09.832873+0000","last_clean":"2024-05-08T22:35:09.832873+0000","last_became_active":"2024-05-08T22:35:09.826639+0000","last_became_peered":"2024-05-08T22:35:09.826639+0000","last_unstale":"2024-05-08T22:35:09.832873+0000","last_undegraded":"2024-05-08T22:35:09.832873+0000","last_fullsized":"2024-05-08T22:35:09.832873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:39.999049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470041+0000","last_change":"2024-05-08T22:35:09.828507+0000","last_active":"2024-05-08T22:35:10.470041+0000","last_peered":"2024-05-08T22:35:10.470041+0000","last_clean":"2024-05-08T22:35:10.470041+0000","last_became_active":"2024-05-08T22:35:09.828076+0000","last_became_peered":"2024-05-08T22:35:09.828076+0000","last_unstale":"2024-05-08T22:35:10.470041+0000","last_undegraded":"2024-05-08T22:35:10.470041+0000","last_fullsized":"2024-05-08T22:35:10.470041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:33.475538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840770+0000","last_change":"2024-05-08T22:35:09.820506+0000","last_active":"2024-05-08T22:35:11.840770+0000","last_peered":"2024-05-08T22:35:11.840770+0000","last_clean":"2024-05-08T22:35:11.840770+0000","last_became_active":"2024-05-08T22:35:09.820208+0000","last_became_peered":"2024-05-08T22:35:09.820208+0000","last_unstale":"2024-05-08T22:35:11.840770+0000","last_undegraded":"2024-05-08T22:35:11.840770+0000","last_fullsized":"2024-05-08T22:35:11.840770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:47:00.753509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.840848+0000","last_change":"2024-05-08T22:35:09.837171+0000","last_active":"2024-05-08T22:35:11.840848+0000","last_peered":"2024-05-08T22:35:11.840848+0000","last_clean":"2024-05-08T22:35:11.840848+0000","last_became_active":"2024-05-08T22:35:09.836290+0000","last_became_peered":"2024-05-08T22:35:09.836290+0000","last_unstale":"2024-05-08T22:35:11.840848+0000","last_undegraded":"2024-05-08T22:35:11.840848+0000","last_fullsized":"2024-05-08T22:35:11.840848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:13.029182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812403+0000","last_change":"2024-05-08T22:35:09.836148+0000","last_active":"2024-05-08T22:35:10.812403+0000","last_peered":"2024-05-08T22:35:10.812403+0000","last_clean":"2024-05-08T22:35:10.812403+0000","last_became_active":"2024-05-08T22:35:09.835767+0000","last_became_peered":"2024-05-08T22:35:09.835767+0000","last_unstale":"2024-05-08T22:35:10.812403+0000","last_undegraded":"2024-05-08T22:35:10.812403+0000","last_fullsized":"2024-05-08T22:35:10.812403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:23.343495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.841116+0000","last_change":"2024-05-08T22:35:09.840558+0000","last_active":"2024-05-08T22:35:09.841116+0000","last_peered":"2024-05-08T22:35:09.841116+0000","last_clean":"2024-05-08T22:35:09.841116+0000","last_became_active":"2024-05-08T22:35:09.840080+0000","last_became_peered":"2024-05-08T22:35:09.840080+0000","last_unstale":"2024-05-08T22:35:09.841116+0000","last_undegraded":"2024-05-08T22:35:09.841116+0000","last_fullsized":"2024-05-08T22:35:09.841116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:32.081217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.842689+0000","last_change":"2024-05-08T22:35:09.831151+0000","last_active":"2024-05-08T22:35:11.842689+0000","last_peered":"2024-05-08T22:35:11.842689+0000","last_clean":"2024-05-08T22:35:11.842689+0000","last_became_active":"2024-05-08T22:35:09.830693+0000","last_became_peered":"2024-05-08T22:35:09.830693+0000","last_unstale":"2024-05-08T22:35:11.842689+0000","last_undegraded":"2024-05-08T22:35:11.842689+0000","last_fullsized":"2024-05-08T22:35:11.842689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:15.060787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841080+0000","last_change":"2024-05-08T22:35:09.830620+0000","last_active":"2024-05-08T22:35:11.841080+0000","last_peered":"2024-05-08T22:35:11.841080+0000","last_clean":"2024-05-08T22:35:11.841080+0000","last_became_active":"2024-05-08T22:35:09.830289+0000","last_became_peered":"2024-05-08T22:35:09.830289+0000","last_unstale":"2024-05-08T22:35:11.841080+0000","last_undegraded":"2024-05-08T22:35:11.841080+0000","last_fullsized":"2024-05-08T22:35:11.841080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:49:15.181957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":54,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:42.328576+0000","last_change":"2024-05-08T22:35:09.391570+0000","last_active":"2024-05-08T22:35:42.328576+0000","last_peered":"2024-05-08T22:35:42.328576+0000","last_clean":"2024-05-08T22:35:42.328576+0000","last_became_active":"2024-05-08T22:35:09.391234+0000","last_became_peered":"2024-05-08T22:35:09.391234+0000","last_unstale":"2024-05-08T22:35:42.328576+0000","last_undegraded":"2024-05-08T22:35:42.328576+0000","last_fullsized":"2024-05-08T22:35:42.328576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:37.226688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825589+0000","last_change":"2024-05-08T22:35:09.831973+0000","last_active":"2024-05-08T22:35:11.825589+0000","last_peered":"2024-05-08T22:35:11.825589+0000","last_clean":"2024-05-08T22:35:11.825589+0000","last_became_active":"2024-05-08T22:35:09.829817+0000","last_became_peered":"2024-05-08T22:35:09.829817+0000","last_unstale":"2024-05-08T22:35:11.825589+0000","last_undegraded":"2024-05-08T22:35:11.825589+0000","last_fullsized":"2024-05-08T22:35:11.825589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:53.829103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841201+0000","last_change":"2024-05-08T22:35:09.393821+0000","last_active":"2024-05-08T22:35:11.841201+0000","last_peered":"2024-05-08T22:35:11.841201+0000","last_clean":"2024-05-08T22:35:11.841201+0000","last_became_active":"2024-05-08T22:35:09.393503+0000","last_became_peered":"2024-05-08T22:35:09.393503+0000","last_unstale":"2024-05-08T22:35:11.841201+0000","last_undegraded":"2024-05-08T22:35:11.841201+0000","last_fullsized":"2024-05-08T22:35:11.841201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:56.252015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841407+0000","last_change":"2024-05-08T22:35:09.823341+0000","last_active":"2024-05-08T22:35:11.841407+0000","last_peered":"2024-05-08T22:35:11.841407+0000","last_clean":"2024-05-08T22:35:11.841407+0000","last_became_active":"2024-05-08T22:35:09.822949+0000","last_became_peered":"2024-05-08T22:35:09.822949+0000","last_unstale":"2024-05-08T22:35:11.841407+0000","last_undegraded":"2024-05-08T22:35:11.841407+0000","last_fullsized":"2024-05-08T22:35:11.841407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:49.189563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.268266+0000","last_change":"2024-05-08T22:35:09.391129+0000","last_active":"2024-05-08T22:35:15.268266+0000","last_peered":"2024-05-08T22:35:15.268266+0000","last_clean":"2024-05-08T22:35:15.268266+0000","last_became_active":"2024-05-08T22:35:09.390429+0000","last_became_peered":"2024-05-08T22:35:09.390429+0000","last_unstale":"2024-05-08T22:35:15.268266+0000","last_undegraded":"2024-05-08T22:35:15.268266+0000","last_fullsized":"2024-05-08T22:35:15.268266+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:59:18.416456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812741+0000","last_change":"2024-05-08T22:35:09.838274+0000","last_active":"2024-05-08T22:35:10.812741+0000","last_peered":"2024-05-08T22:35:10.812741+0000","last_clean":"2024-05-08T22:35:10.812741+0000","last_became_active":"2024-05-08T22:35:09.837637+0000","last_became_peered":"2024-05-08T22:35:09.837637+0000","last_unstale":"2024-05-08T22:35:10.812741+0000","last_undegraded":"2024-05-08T22:35:10.812741+0000","last_fullsized":"2024-05-08T22:35:10.812741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:00.476914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267817+0000","last_change":"2024-05-08T22:35:09.391987+0000","last_active":"2024-05-08T22:35:15.267817+0000","last_peered":"2024-05-08T22:35:15.267817+0000","last_clean":"2024-05-08T22:35:15.267817+0000","last_became_active":"2024-05-08T22:35:09.390587+0000","last_became_peered":"2024-05-08T22:35:09.390587+0000","last_unstale":"2024-05-08T22:35:15.267817+0000","last_undegraded":"2024-05-08T22:35:15.267817+0000","last_fullsized":"2024-05-08T22:35:15.267817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:14.343341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813185+0000","last_change":"2024-05-08T22:35:09.823177+0000","last_active":"2024-05-08T22:35:10.813185+0000","last_peered":"2024-05-08T22:35:10.813185+0000","last_clean":"2024-05-08T22:35:10.813185+0000","last_became_active":"2024-05-08T22:35:09.822830+0000","last_became_peered":"2024-05-08T22:35:09.822830+0000","last_unstale":"2024-05-08T22:35:10.813185+0000","last_undegraded":"2024-05-08T22:35:10.813185+0000","last_fullsized":"2024-05-08T22:35:10.813185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:44.648818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":53,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:45.309116+0000","last_change":"2024-05-08T22:35:09.392558+0000","last_active":"2024-05-08T22:35:45.309116+0000","last_peered":"2024-05-08T22:35:45.309116+0000","last_clean":"2024-05-08T22:35:45.309116+0000","last_became_active":"2024-05-08T22:35:09.392235+0000","last_became_peered":"2024-05-08T22:35:09.392235+0000","last_unstale":"2024-05-08T22:35:45.309116+0000","last_undegraded":"2024-05-08T22:35:45.309116+0000","last_fullsized":"2024-05-08T22:35:45.309116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:47:12.186400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.830402+0000","last_change":"2024-05-08T22:35:09.823455+0000","last_active":"2024-05-08T22:35:09.830402+0000","last_peered":"2024-05-08T22:35:09.830402+0000","last_clean":"2024-05-08T22:35:09.830402+0000","last_became_active":"2024-05-08T22:35:09.822818+0000","last_became_peered":"2024-05-08T22:35:09.822818+0000","last_unstale":"2024-05-08T22:35:09.830402+0000","last_undegraded":"2024-05-08T22:35:09.830402+0000","last_fullsized":"2024-05-08T22:35:09.830402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:38.142265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:14.857485+0000","last_change":"2024-05-08T22:35:09.394713+0000","last_active":"2024-05-08T22:35:14.857485+0000","last_peered":"2024-05-08T22:35:14.857485+0000","last_clean":"2024-05-08T22:35:14.857485+0000","last_became_active":"2024-05-08T22:35:09.394397+0000","last_became_peered":"2024-05-08T22:35:09.394397+0000","last_unstale":"2024-05-08T22:35:14.857485+0000","last_undegraded":"2024-05-08T22:35:14.857485+0000","last_fullsized":"2024-05-08T22:35:14.857485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:58:30.594207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813065+0000","last_change":"2024-05-08T22:35:09.827921+0000","last_active":"2024-05-08T22:35:10.813065+0000","last_peered":"2024-05-08T22:35:10.813065+0000","last_clean":"2024-05-08T22:35:10.813065+0000","last_became_active":"2024-05-08T22:35:09.827387+0000","last_became_peered":"2024-05-08T22:35:09.827387+0000","last_unstale":"2024-05-08T22:35:10.813065+0000","last_undegraded":"2024-05-08T22:35:10.813065+0000","last_fullsized":"2024-05-08T22:35:10.813065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:33.986935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.845741+0000","last_change":"2024-05-08T22:35:09.384387+0000","last_active":"2024-05-08T22:35:13.845741+0000","last_peered":"2024-05-08T22:35:13.845741+0000","last_clean":"2024-05-08T22:35:13.845741+0000","last_became_active":"2024-05-08T22:35:09.383970+0000","last_became_peered":"2024-05-08T22:35:09.383970+0000","last_unstale":"2024-05-08T22:35:13.845741+0000","last_undegraded":"2024-05-08T22:35:13.845741+0000","last_fullsized":"2024-05-08T22:35:13.845741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:55.430960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825444+0000","last_change":"2024-05-08T22:35:09.831477+0000","last_active":"2024-05-08T22:35:11.825444+0000","last_peered":"2024-05-08T22:35:11.825444+0000","last_clean":"2024-05-08T22:35:11.825444+0000","last_became_active":"2024-05-08T22:35:09.831168+0000","last_became_peered":"2024-05-08T22:35:09.831168+0000","last_unstale":"2024-05-08T22:35:11.825444+0000","last_undegraded":"2024-05-08T22:35:11.825444+0000","last_fullsized":"2024-05-08T22:35:11.825444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:56.765210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267651+0000","last_change":"2024-05-08T22:35:09.389063+0000","last_active":"2024-05-08T22:35:15.267651+0000","last_peered":"2024-05-08T22:35:15.267651+0000","last_clean":"2024-05-08T22:35:15.267651+0000","last_became_active":"2024-05-08T22:35:09.388683+0000","last_became_peered":"2024-05-08T22:35:09.388683+0000","last_unstale":"2024-05-08T22:35:15.267651+0000","last_undegraded":"2024-05-08T22:35:15.267651+0000","last_fullsized":"2024-05-08T22:35:15.267651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:03.804688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.842741+0000","last_change":"2024-05-08T22:35:09.829277+0000","last_active":"2024-05-08T22:35:11.842741+0000","last_peered":"2024-05-08T22:35:11.842741+0000","last_clean":"2024-05-08T22:35:11.842741+0000","last_became_active":"2024-05-08T22:35:09.828782+0000","last_became_peered":"2024-05-08T22:35:09.828782+0000","last_unstale":"2024-05-08T22:35:11.842741+0000","last_undegraded":"2024-05-08T22:35:11.842741+0000","last_fullsized":"2024-05-08T22:35:11.842741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:47.012528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":80,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:46.310098+0000","last_change":"2024-05-08T22:35:09.387464+0000","last_active":"2024-05-08T22:35:46.310098+0000","last_peered":"2024-05-08T22:35:46.310098+0000","last_clean":"2024-05-08T22:35:46.310098+0000","last_became_active":"2024-05-08T22:35:09.386994+0000","last_became_peered":"2024-05-08T22:35:09.386994+0000","last_unstale":"2024-05-08T22:35:46.310098+0000","last_undegraded":"2024-05-08T22:35:46.310098+0000","last_fullsized":"2024-05-08T22:35:46.310098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:27:04.151479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812918+0000","last_change":"2024-05-08T22:35:09.830886+0000","last_active":"2024-05-08T22:35:10.812918+0000","last_peered":"2024-05-08T22:35:10.812918+0000","last_clean":"2024-05-08T22:35:10.812918+0000","last_became_active":"2024-05-08T22:35:09.830466+0000","last_became_peered":"2024-05-08T22:35:09.830466+0000","last_unstale":"2024-05-08T22:35:10.812918+0000","last_undegraded":"2024-05-08T22:35:10.812918+0000","last_fullsized":"2024-05-08T22:35:10.812918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:12.337934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267954+0000","last_change":"2024-05-08T22:35:09.391510+0000","last_active":"2024-05-08T22:35:15.267954+0000","last_peered":"2024-05-08T22:35:15.267954+0000","last_clean":"2024-05-08T22:35:15.267954+0000","last_became_active":"2024-05-08T22:35:09.391145+0000","last_became_peered":"2024-05-08T22:35:09.391145+0000","last_unstale":"2024-05-08T22:35:15.267954+0000","last_undegraded":"2024-05-08T22:35:15.267954+0000","last_fullsized":"2024-05-08T22:35:15.267954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:39.694122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841304+0000","last_change":"2024-05-08T22:35:09.837056+0000","last_active":"2024-05-08T22:35:11.841304+0000","last_peered":"2024-05-08T22:35:11.841304+0000","last_clean":"2024-05-08T22:35:11.841304+0000","last_became_active":"2024-05-08T22:35:09.836724+0000","last_became_peered":"2024-05-08T22:35:09.836724+0000","last_unstale":"2024-05-08T22:35:11.841304+0000","last_undegraded":"2024-05-08T22:35:11.841304+0000","last_fullsized":"2024-05-08T22:35:11.841304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:48.901289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:14.855791+0000","last_change":"2024-05-08T22:35:09.395826+0000","last_active":"2024-05-08T22:35:14.855791+0000","last_peered":"2024-05-08T22:35:14.855791+0000","last_clean":"2024-05-08T22:35:14.855791+0000","last_became_active":"2024-05-08T22:35:09.395330+0000","last_became_peered":"2024-05-08T22:35:09.395330+0000","last_unstale":"2024-05-08T22:35:14.855791+0000","last_undegraded":"2024-05-08T22:35:14.855791+0000","last_fullsized":"2024-05-08T22:35:14.855791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:43.577894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812839+0000","last_change":"2024-05-08T22:35:09.831625+0000","last_active":"2024-05-08T22:35:10.812839+0000","last_peered":"2024-05-08T22:35:10.812839+0000","last_clean":"2024-05-08T22:35:10.812839+0000","last_became_active":"2024-05-08T22:35:09.831310+0000","last_became_peered":"2024-05-08T22:35:09.831310+0000","last_unstale":"2024-05-08T22:35:10.812839+0000","last_undegraded":"2024-05-08T22:35:10.812839+0000","last_fullsized":"2024-05-08T22:35:10.812839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:16.743871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'29","reported_seq":112,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:46.317738+0000","last_change":"2024-05-08T22:35:09.389805+0000","last_active":"2024-05-08T22:35:46.317738+0000","last_peered":"2024-05-08T22:35:46.317738+0000","last_clean":"2024-05-08T22:35:46.317738+0000","last_became_active":"2024-05-08T22:35:09.389443+0000","last_became_peered":"2024-05-08T22:35:09.389443+0000","last_unstale":"2024-05-08T22:35:46.317738+0000","last_undegraded":"2024-05-08T22:35:46.317738+0000","last_fullsized":"2024-05-08T22:35:46.317738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:40.479503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59419,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":71,"num_read_kb":93,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.835300+0000","last_change":"2024-05-08T22:35:09.834871+0000","last_active":"2024-05-08T22:35:09.835300+0000","last_peered":"2024-05-08T22:35:09.835300+0000","last_clean":"2024-05-08T22:35:09.835300+0000","last_became_active":"2024-05-08T22:35:09.834589+0000","last_became_peered":"2024-05-08T22:35:09.834589+0000","last_unstale":"2024-05-08T22:35:09.835300+0000","last_undegraded":"2024-05-08T22:35:09.835300+0000","last_fullsized":"2024-05-08T22:35:09.835300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:12.476142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267144+0000","last_change":"2024-05-08T22:35:09.388615+0000","last_active":"2024-05-08T22:35:15.267144+0000","last_peered":"2024-05-08T22:35:15.267144+0000","last_clean":"2024-05-08T22:35:15.267144+0000","last_became_active":"2024-05-08T22:35:09.388272+0000","last_became_peered":"2024-05-08T22:35:09.388272+0000","last_unstale":"2024-05-08T22:35:15.267144+0000","last_undegraded":"2024-05-08T22:35:15.267144+0000","last_fullsized":"2024-05-08T22:35:15.267144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:23.565450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470024+0000","last_change":"2024-05-08T22:35:09.831511+0000","last_active":"2024-05-08T22:35:10.470024+0000","last_peered":"2024-05-08T22:35:10.470024+0000","last_clean":"2024-05-08T22:35:10.470024+0000","last_became_active":"2024-05-08T22:35:09.831216+0000","last_became_peered":"2024-05-08T22:35:09.831216+0000","last_unstale":"2024-05-08T22:35:10.470024+0000","last_undegraded":"2024-05-08T22:35:10.470024+0000","last_fullsized":"2024-05-08T22:35:10.470024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:46.244544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.268190+0000","last_change":"2024-05-08T22:35:09.391835+0000","last_active":"2024-05-08T22:35:15.268190+0000","last_peered":"2024-05-08T22:35:15.268190+0000","last_clean":"2024-05-08T22:35:15.268190+0000","last_became_active":"2024-05-08T22:35:09.391465+0000","last_became_peered":"2024-05-08T22:35:09.391465+0000","last_unstale":"2024-05-08T22:35:15.268190+0000","last_undegraded":"2024-05-08T22:35:15.268190+0000","last_fullsized":"2024-05-08T22:35:15.268190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:30.819323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623032+0000","last_change":"2024-05-08T22:35:09.840351+0000","last_active":"2024-05-08T22:35:11.623032+0000","last_peered":"2024-05-08T22:35:11.623032+0000","last_clean":"2024-05-08T22:35:11.623032+0000","last_became_active":"2024-05-08T22:35:09.840000+0000","last_became_peered":"2024-05-08T22:35:09.840000+0000","last_unstale":"2024-05-08T22:35:11.623032+0000","last_undegraded":"2024-05-08T22:35:11.623032+0000","last_fullsized":"2024-05-08T22:35:11.623032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:51.428513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":51,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:43.313818+0000","last_change":"2024-05-08T22:35:09.389572+0000","last_active":"2024-05-08T22:35:43.313818+0000","last_peered":"2024-05-08T22:35:43.313818+0000","last_clean":"2024-05-08T22:35:43.313818+0000","last_became_active":"2024-05-08T22:35:09.389245+0000","last_became_peered":"2024-05-08T22:35:09.389245+0000","last_unstale":"2024-05-08T22:35:43.313818+0000","last_undegraded":"2024-05-08T22:35:43.313818+0000","last_fullsized":"2024-05-08T22:35:43.313818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:04.992010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.622950+0000","last_change":"2024-05-08T22:35:09.840132+0000","last_active":"2024-05-08T22:35:11.622950+0000","last_peered":"2024-05-08T22:35:11.622950+0000","last_clean":"2024-05-08T22:35:11.622950+0000","last_became_active":"2024-05-08T22:35:09.839684+0000","last_became_peered":"2024-05-08T22:35:09.839684+0000","last_unstale":"2024-05-08T22:35:11.622950+0000","last_undegraded":"2024-05-08T22:35:11.622950+0000","last_fullsized":"2024-05-08T22:35:11.622950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:32.325406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.258306+0000","last_change":"2024-05-08T22:35:09.392220+0000","last_active":"2024-05-08T22:35:15.258306+0000","last_peered":"2024-05-08T22:35:15.258306+0000","last_clean":"2024-05-08T22:35:15.258306+0000","last_became_active":"2024-05-08T22:35:09.391889+0000","last_became_peered":"2024-05-08T22:35:09.391889+0000","last_unstale":"2024-05-08T22:35:15.258306+0000","last_undegraded":"2024-05-08T22:35:15.258306+0000","last_fullsized":"2024-05-08T22:35:15.258306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:01.417894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813015+0000","last_change":"2024-05-08T22:35:09.823395+0000","last_active":"2024-05-08T22:35:10.813015+0000","last_peered":"2024-05-08T22:35:10.813015+0000","last_clean":"2024-05-08T22:35:10.813015+0000","last_became_active":"2024-05-08T22:35:09.822484+0000","last_became_peered":"2024-05-08T22:35:09.822484+0000","last_unstale":"2024-05-08T22:35:10.813015+0000","last_undegraded":"2024-05-08T22:35:10.813015+0000","last_fullsized":"2024-05-08T22:35:10.813015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:35.330789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.257929+0000","last_change":"2024-05-08T22:35:09.379904+0000","last_active":"2024-05-08T22:35:15.257929+0000","last_peered":"2024-05-08T22:35:15.257929+0000","last_clean":"2024-05-08T22:35:15.257929+0000","last_became_active":"2024-05-08T22:35:09.379533+0000","last_became_peered":"2024-05-08T22:35:09.379533+0000","last_unstale":"2024-05-08T22:35:15.257929+0000","last_undegraded":"2024-05-08T22:35:15.257929+0000","last_fullsized":"2024-05-08T22:35:15.257929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:14.625069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.838589+0000","last_change":"2024-05-08T22:35:09.836472+0000","last_active":"2024-05-08T22:35:09.838589+0000","last_peered":"2024-05-08T22:35:09.838589+0000","last_clean":"2024-05-08T22:35:09.838589+0000","last_became_active":"2024-05-08T22:35:09.836110+0000","last_became_peered":"2024-05-08T22:35:09.836110+0000","last_unstale":"2024-05-08T22:35:09.838589+0000","last_undegraded":"2024-05-08T22:35:09.838589+0000","last_fullsized":"2024-05-08T22:35:09.838589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:48.979661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.842537+0000","last_change":"2024-05-08T22:35:09.391647+0000","last_active":"2024-05-08T22:35:11.842537+0000","last_peered":"2024-05-08T22:35:11.842537+0000","last_clean":"2024-05-08T22:35:11.842537+0000","last_became_active":"2024-05-08T22:35:09.391273+0000","last_became_peered":"2024-05-08T22:35:09.391273+0000","last_unstale":"2024-05-08T22:35:11.842537+0000","last_undegraded":"2024-05-08T22:35:11.842537+0000","last_fullsized":"2024-05-08T22:35:11.842537+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:48.001313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.752901+0000","last_change":"2024-05-08T22:35:09.828713+0000","last_active":"2024-05-08T22:35:11.752901+0000","last_peered":"2024-05-08T22:35:11.752901+0000","last_clean":"2024-05-08T22:35:11.752901+0000","last_became_active":"2024-05-08T22:35:09.828370+0000","last_became_peered":"2024-05-08T22:35:09.828370+0000","last_unstale":"2024-05-08T22:35:11.752901+0000","last_undegraded":"2024-05-08T22:35:11.752901+0000","last_fullsized":"2024-05-08T22:35:11.752901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:23.328029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":49,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:45.332958+0000","last_change":"2024-05-08T22:35:09.391658+0000","last_active":"2024-05-08T22:35:45.332958+0000","last_peered":"2024-05-08T22:35:45.332958+0000","last_clean":"2024-05-08T22:35:45.332958+0000","last_became_active":"2024-05-08T22:35:09.391326+0000","last_became_peered":"2024-05-08T22:35:09.391326+0000","last_unstale":"2024-05-08T22:35:45.332958+0000","last_undegraded":"2024-05-08T22:35:45.332958+0000","last_fullsized":"2024-05-08T22:35:45.332958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:08:13.933145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.604425+0000","last_change":"2024-05-08T22:35:09.837816+0000","last_active":"2024-05-08T22:35:11.604425+0000","last_peered":"2024-05-08T22:35:11.604425+0000","last_clean":"2024-05-08T22:35:11.604425+0000","last_became_active":"2024-05-08T22:35:09.837471+0000","last_became_peered":"2024-05-08T22:35:09.837471+0000","last_unstale":"2024-05-08T22:35:11.604425+0000","last_undegraded":"2024-05-08T22:35:11.604425+0000","last_fullsized":"2024-05-08T22:35:11.604425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:04.564055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.813006+0000","last_change":"2024-05-08T22:35:09.827092+0000","last_active":"2024-05-08T22:35:10.813006+0000","last_peered":"2024-05-08T22:35:10.813006+0000","last_clean":"2024-05-08T22:35:10.813006+0000","last_became_active":"2024-05-08T22:35:09.826238+0000","last_became_peered":"2024-05-08T22:35:09.826238+0000","last_unstale":"2024-05-08T22:35:10.813006+0000","last_undegraded":"2024-05-08T22:35:10.813006+0000","last_fullsized":"2024-05-08T22:35:10.813006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:26:59.504555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":49,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:44.314446+0000","last_change":"2024-05-08T22:35:09.391555+0000","last_active":"2024-05-08T22:35:44.314446+0000","last_peered":"2024-05-08T22:35:44.314446+0000","last_clean":"2024-05-08T22:35:44.314446+0000","last_became_active":"2024-05-08T22:35:09.391294+0000","last_became_peered":"2024-05-08T22:35:09.391294+0000","last_unstale":"2024-05-08T22:35:44.314446+0000","last_undegraded":"2024-05-08T22:35:44.314446+0000","last_fullsized":"2024-05-08T22:35:44.314446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:41.195882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825544+0000","last_change":"2024-05-08T22:35:09.822814+0000","last_active":"2024-05-08T22:35:11.825544+0000","last_peered":"2024-05-08T22:35:11.825544+0000","last_clean":"2024-05-08T22:35:11.825544+0000","last_became_active":"2024-05-08T22:35:09.822425+0000","last_became_peered":"2024-05-08T22:35:09.822425+0000","last_unstale":"2024-05-08T22:35:11.825544+0000","last_undegraded":"2024-05-08T22:35:11.825544+0000","last_fullsized":"2024-05-08T22:35:11.825544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:26:56.212615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.265500+0000","last_change":"2024-05-08T22:35:09.383782+0000","last_active":"2024-05-08T22:35:15.265500+0000","last_peered":"2024-05-08T22:35:15.265500+0000","last_clean":"2024-05-08T22:35:15.265500+0000","last_became_active":"2024-05-08T22:35:09.383285+0000","last_became_peered":"2024-05-08T22:35:09.383285+0000","last_unstale":"2024-05-08T22:35:15.265500+0000","last_undegraded":"2024-05-08T22:35:15.265500+0000","last_fullsized":"2024-05-08T22:35:15.265500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:42.380683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":323,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841287+0000","last_change":"2024-05-08T22:34:20.037406+0000","last_active":"2024-05-08T22:35:11.841287+0000","last_peered":"2024-05-08T22:35:11.841287+0000","last_clean":"2024-05-08T22:35:11.841287+0000","last_became_active":"2024-05-08T22:34:20.036880+0000","last_became_peered":"2024-05-08T22:34:20.036880+0000","last_unstale":"2024-05-08T22:35:11.841287+0000","last_undegraded":"2024-05-08T22:35:11.841287+0000","last_fullsized":"2024-05-08T22:35:11.841287+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:31:54.953551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:31:54.953551+0000","last_clean_scrub_stamp":"2024-05-08T22:31:54.953551+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:00.820169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.469956+0000","last_change":"2024-05-08T22:35:09.830425+0000","last_active":"2024-05-08T22:35:10.469956+0000","last_peered":"2024-05-08T22:35:10.469956+0000","last_clean":"2024-05-08T22:35:10.469956+0000","last_became_active":"2024-05-08T22:35:09.830097+0000","last_became_peered":"2024-05-08T22:35:09.830097+0000","last_unstale":"2024-05-08T22:35:10.469956+0000","last_undegraded":"2024-05-08T22:35:10.469956+0000","last_fullsized":"2024-05-08T22:35:10.469956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:53.445798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:13.846273+0000","last_change":"2024-05-08T22:35:09.386717+0000","last_active":"2024-05-08T22:35:13.846273+0000","last_peered":"2024-05-08T22:35:13.846273+0000","last_clean":"2024-05-08T22:35:13.846273+0000","last_became_active":"2024-05-08T22:35:09.386371+0000","last_became_peered":"2024-05-08T22:35:09.386371+0000","last_unstale":"2024-05-08T22:35:13.846273+0000","last_undegraded":"2024-05-08T22:35:13.846273+0000","last_fullsized":"2024-05-08T22:35:13.846273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:22:41.061997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.257951+0000","last_change":"2024-05-08T22:35:09.394542+0000","last_active":"2024-05-08T22:35:15.257951+0000","last_peered":"2024-05-08T22:35:15.257951+0000","last_clean":"2024-05-08T22:35:15.257951+0000","last_became_active":"2024-05-08T22:35:09.394223+0000","last_became_peered":"2024-05-08T22:35:09.394223+0000","last_unstale":"2024-05-08T22:35:15.257951+0000","last_undegraded":"2024-05-08T22:35:15.257951+0000","last_fullsized":"2024-05-08T22:35:15.257951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:02.162377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470081+0000","last_change":"2024-05-08T22:35:09.825810+0000","last_active":"2024-05-08T22:35:10.470081+0000","last_peered":"2024-05-08T22:35:10.470081+0000","last_clean":"2024-05-08T22:35:10.470081+0000","last_became_active":"2024-05-08T22:35:09.825516+0000","last_became_peered":"2024-05-08T22:35:09.825516+0000","last_unstale":"2024-05-08T22:35:10.470081+0000","last_undegraded":"2024-05-08T22:35:10.470081+0000","last_fullsized":"2024-05-08T22:35:10.470081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:24.087187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'9","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:19.069957+0000","last_change":"2024-05-08T22:35:09.381688+0000","last_active":"2024-05-08T22:35:19.069957+0000","last_peered":"2024-05-08T22:35:19.069957+0000","last_clean":"2024-05-08T22:35:19.069957+0000","last_became_active":"2024-05-08T22:35:09.381343+0000","last_became_peered":"2024-05-08T22:35:09.381343+0000","last_unstale":"2024-05-08T22:35:19.069957+0000","last_undegraded":"2024-05-08T22:35:19.069957+0000","last_fullsized":"2024-05-08T22:35:19.069957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:14:16.142265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825676+0000","last_change":"2024-05-08T22:35:09.831689+0000","last_active":"2024-05-08T22:35:11.825676+0000","last_peered":"2024-05-08T22:35:11.825676+0000","last_clean":"2024-05-08T22:35:11.825676+0000","last_became_active":"2024-05-08T22:35:09.831337+0000","last_became_peered":"2024-05-08T22:35:09.831337+0000","last_unstale":"2024-05-08T22:35:11.825676+0000","last_undegraded":"2024-05-08T22:35:11.825676+0000","last_fullsized":"2024-05-08T22:35:11.825676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.965340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267698+0000","last_change":"2024-05-08T22:35:09.390479+0000","last_active":"2024-05-08T22:35:15.267698+0000","last_peered":"2024-05-08T22:35:15.267698+0000","last_clean":"2024-05-08T22:35:15.267698+0000","last_became_active":"2024-05-08T22:35:09.389790+0000","last_became_peered":"2024-05-08T22:35:09.389790+0000","last_unstale":"2024-05-08T22:35:15.267698+0000","last_undegraded":"2024-05-08T22:35:15.267698+0000","last_fullsized":"2024-05-08T22:35:15.267698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:27.668801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841685+0000","last_change":"2024-05-08T22:35:09.836123+0000","last_active":"2024-05-08T22:35:11.841685+0000","last_peered":"2024-05-08T22:35:11.841685+0000","last_clean":"2024-05-08T22:35:11.841685+0000","last_became_active":"2024-05-08T22:35:09.835840+0000","last_became_peered":"2024-05-08T22:35:09.835840+0000","last_unstale":"2024-05-08T22:35:11.841685+0000","last_undegraded":"2024-05-08T22:35:11.841685+0000","last_fullsized":"2024-05-08T22:35:11.841685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:52.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.840918+0000","last_change":"2024-05-08T22:35:09.394658+0000","last_active":"2024-05-08T22:35:12.840918+0000","last_peered":"2024-05-08T22:35:12.840918+0000","last_clean":"2024-05-08T22:35:12.840918+0000","last_became_active":"2024-05-08T22:35:09.394351+0000","last_became_peered":"2024-05-08T22:35:09.394351+0000","last_unstale":"2024-05-08T22:35:12.840918+0000","last_undegraded":"2024-05-08T22:35:12.840918+0000","last_fullsized":"2024-05-08T22:35:12.840918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:32.954612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.834139+0000","last_change":"2024-05-08T22:35:09.830853+0000","last_active":"2024-05-08T22:35:09.834139+0000","last_peered":"2024-05-08T22:35:09.834139+0000","last_clean":"2024-05-08T22:35:09.834139+0000","last_became_active":"2024-05-08T22:35:09.830471+0000","last_became_peered":"2024-05-08T22:35:09.830471+0000","last_unstale":"2024-05-08T22:35:09.834139+0000","last_undegraded":"2024-05-08T22:35:09.834139+0000","last_fullsized":"2024-05-08T22:35:09.834139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:13.960966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.265416+0000","last_change":"2024-05-08T22:35:09.382515+0000","last_active":"2024-05-08T22:35:15.265416+0000","last_peered":"2024-05-08T22:35:15.265416+0000","last_clean":"2024-05-08T22:35:15.265416+0000","last_became_active":"2024-05-08T22:35:09.382168+0000","last_became_peered":"2024-05-08T22:35:09.382168+0000","last_unstale":"2024-05-08T22:35:15.265416+0000","last_undegraded":"2024-05-08T22:35:15.265416+0000","last_fullsized":"2024-05-08T22:35:15.265416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:10.248542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.837958+0000","last_change":"2024-05-08T22:35:09.835925+0000","last_active":"2024-05-08T22:35:09.837958+0000","last_peered":"2024-05-08T22:35:09.837958+0000","last_clean":"2024-05-08T22:35:09.837958+0000","last_became_active":"2024-05-08T22:35:09.835547+0000","last_became_peered":"2024-05-08T22:35:09.835547+0000","last_unstale":"2024-05-08T22:35:09.837958+0000","last_undegraded":"2024-05-08T22:35:09.837958+0000","last_fullsized":"2024-05-08T22:35:09.837958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:36.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:14.857829+0000","last_change":"2024-05-08T22:35:09.385035+0000","last_active":"2024-05-08T22:35:14.857829+0000","last_peered":"2024-05-08T22:35:14.857829+0000","last_clean":"2024-05-08T22:35:14.857829+0000","last_became_active":"2024-05-08T22:35:09.384274+0000","last_became_peered":"2024-05-08T22:35:09.384274+0000","last_unstale":"2024-05-08T22:35:14.857829+0000","last_undegraded":"2024-05-08T22:35:14.857829+0000","last_fullsized":"2024-05-08T22:35:14.857829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:50.654985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825567+0000","last_change":"2024-05-08T22:35:09.821454+0000","last_active":"2024-05-08T22:35:11.825567+0000","last_peered":"2024-05-08T22:35:11.825567+0000","last_clean":"2024-05-08T22:35:11.825567+0000","last_became_active":"2024-05-08T22:35:09.821199+0000","last_became_peered":"2024-05-08T22:35:09.821199+0000","last_unstale":"2024-05-08T22:35:11.825567+0000","last_undegraded":"2024-05-08T22:35:11.825567+0000","last_fullsized":"2024-05-08T22:35:11.825567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:02.677307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:12.840831+0000","last_change":"2024-05-08T22:35:09.394027+0000","last_active":"2024-05-08T22:35:12.840831+0000","last_peered":"2024-05-08T22:35:12.840831+0000","last_clean":"2024-05-08T22:35:12.840831+0000","last_became_active":"2024-05-08T22:35:09.393622+0000","last_became_peered":"2024-05-08T22:35:09.393622+0000","last_unstale":"2024-05-08T22:35:12.840831+0000","last_undegraded":"2024-05-08T22:35:12.840831+0000","last_fullsized":"2024-05-08T22:35:12.840831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:13.415635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623012+0000","last_change":"2024-05-08T22:35:09.842928+0000","last_active":"2024-05-08T22:35:11.623012+0000","last_peered":"2024-05-08T22:35:11.623012+0000","last_clean":"2024-05-08T22:35:11.623012+0000","last_became_active":"2024-05-08T22:35:09.842576+0000","last_became_peered":"2024-05-08T22:35:09.842576+0000","last_unstale":"2024-05-08T22:35:11.623012+0000","last_undegraded":"2024-05-08T22:35:11.623012+0000","last_fullsized":"2024-05-08T22:35:11.623012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:03.526259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":48,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:42.304497+0000","last_change":"2024-05-08T22:35:09.396325+0000","last_active":"2024-05-08T22:35:42.304497+0000","last_peered":"2024-05-08T22:35:42.304497+0000","last_clean":"2024-05-08T22:35:42.304497+0000","last_became_active":"2024-05-08T22:35:09.395656+0000","last_became_peered":"2024-05-08T22:35:09.395656+0000","last_unstale":"2024-05-08T22:35:42.304497+0000","last_undegraded":"2024-05-08T22:35:42.304497+0000","last_fullsized":"2024-05-08T22:35:42.304497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:30.326694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623076+0000","last_change":"2024-05-08T22:35:09.840682+0000","last_active":"2024-05-08T22:35:11.623076+0000","last_peered":"2024-05-08T22:35:11.623076+0000","last_clean":"2024-05-08T22:35:11.623076+0000","last_became_active":"2024-05-08T22:35:09.840315+0000","last_became_peered":"2024-05-08T22:35:09.840315+0000","last_unstale":"2024-05-08T22:35:11.623076+0000","last_undegraded":"2024-05-08T22:35:11.623076+0000","last_fullsized":"2024-05-08T22:35:11.623076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:13.884569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.257932+0000","last_change":"2024-05-08T22:35:09.389169+0000","last_active":"2024-05-08T22:35:15.257932+0000","last_peered":"2024-05-08T22:35:15.257932+0000","last_clean":"2024-05-08T22:35:15.257932+0000","last_became_active":"2024-05-08T22:35:09.388836+0000","last_became_peered":"2024-05-08T22:35:09.388836+0000","last_unstale":"2024-05-08T22:35:15.257932+0000","last_undegraded":"2024-05-08T22:35:15.257932+0000","last_fullsized":"2024-05-08T22:35:15.257932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:08.960719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841499+0000","last_change":"2024-05-08T22:35:09.833885+0000","last_active":"2024-05-08T22:35:11.841499+0000","last_peered":"2024-05-08T22:35:11.841499+0000","last_clean":"2024-05-08T22:35:11.841499+0000","last_became_active":"2024-05-08T22:35:09.833596+0000","last_became_peered":"2024-05-08T22:35:09.833596+0000","last_unstale":"2024-05-08T22:35:11.841499+0000","last_undegraded":"2024-05-08T22:35:11.841499+0000","last_fullsized":"2024-05-08T22:35:11.841499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:21:29.673837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:15.267349+0000","last_change":"2024-05-08T22:35:09.389023+0000","last_active":"2024-05-08T22:35:15.267349+0000","last_peered":"2024-05-08T22:35:15.267349+0000","last_clean":"2024-05-08T22:35:15.267349+0000","last_became_active":"2024-05-08T22:35:09.388721+0000","last_became_peered":"2024-05-08T22:35:09.388721+0000","last_unstale":"2024-05-08T22:35:15.267349+0000","last_undegraded":"2024-05-08T22:35:15.267349+0000","last_fullsized":"2024-05-08T22:35:15.267349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:08.361698+0000","last_clean_scrub_stamp":"2024-05-08T22:35:08.361698+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:41.776592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.623109+0000","last_change":"2024-05-08T22:35:09.839499+0000","last_active":"2024-05-08T22:35:11.623109+0000","last_peered":"2024-05-08T22:35:11.623109+0000","last_clean":"2024-05-08T22:35:11.623109+0000","last_became_active":"2024-05-08T22:35:09.839146+0000","last_became_peered":"2024-05-08T22:35:09.839146+0000","last_unstale":"2024-05-08T22:35:11.623109+0000","last_undegraded":"2024-05-08T22:35:11.623109+0000","last_fullsized":"2024-05-08T22:35:11.623109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:45.636827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.839015+0000","last_change":"2024-05-08T22:35:09.836710+0000","last_active":"2024-05-08T22:35:09.839015+0000","last_peered":"2024-05-08T22:35:09.839015+0000","last_clean":"2024-05-08T22:35:09.839015+0000","last_became_active":"2024-05-08T22:35:09.835838+0000","last_became_peered":"2024-05-08T22:35:09.835838+0000","last_unstale":"2024-05-08T22:35:09.839015+0000","last_undegraded":"2024-05-08T22:35:09.839015+0000","last_fullsized":"2024-05-08T22:35:09.839015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:00.567805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841269+0000","last_change":"2024-05-08T22:35:09.830148+0000","last_active":"2024-05-08T22:35:11.841269+0000","last_peered":"2024-05-08T22:35:11.841269+0000","last_clean":"2024-05-08T22:35:11.841269+0000","last_became_active":"2024-05-08T22:35:09.829820+0000","last_became_peered":"2024-05-08T22:35:09.829820+0000","last_unstale":"2024-05-08T22:35:11.841269+0000","last_undegraded":"2024-05-08T22:35:11.841269+0000","last_fullsized":"2024-05-08T22:35:11.841269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:38.840993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841227+0000","last_change":"2024-05-08T22:35:09.830428+0000","last_active":"2024-05-08T22:35:11.841227+0000","last_peered":"2024-05-08T22:35:11.841227+0000","last_clean":"2024-05-08T22:35:11.841227+0000","last_became_active":"2024-05-08T22:35:09.829736+0000","last_became_peered":"2024-05-08T22:35:09.829736+0000","last_unstale":"2024-05-08T22:35:11.841227+0000","last_undegraded":"2024-05-08T22:35:11.841227+0000","last_fullsized":"2024-05-08T22:35:11.841227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:54.458662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841649+0000","last_change":"2024-05-08T22:35:09.836200+0000","last_active":"2024-05-08T22:35:11.841649+0000","last_peered":"2024-05-08T22:35:11.841649+0000","last_clean":"2024-05-08T22:35:11.841649+0000","last_became_active":"2024-05-08T22:35:09.835892+0000","last_became_peered":"2024-05-08T22:35:09.835892+0000","last_unstale":"2024-05-08T22:35:11.841649+0000","last_undegraded":"2024-05-08T22:35:11.841649+0000","last_fullsized":"2024-05-08T22:35:11.841649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:54:42.487982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470102+0000","last_change":"2024-05-08T22:35:09.834026+0000","last_active":"2024-05-08T22:35:10.470102+0000","last_peered":"2024-05-08T22:35:10.470102+0000","last_clean":"2024-05-08T22:35:10.470102+0000","last_became_active":"2024-05-08T22:35:09.833659+0000","last_became_peered":"2024-05-08T22:35:09.833659+0000","last_unstale":"2024-05-08T22:35:10.470102+0000","last_undegraded":"2024-05-08T22:35:10.470102+0000","last_fullsized":"2024-05-08T22:35:10.470102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:00.536853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.839061+0000","last_change":"2024-05-08T22:35:09.836928+0000","last_active":"2024-05-08T22:35:09.839061+0000","last_peered":"2024-05-08T22:35:09.839061+0000","last_clean":"2024-05-08T22:35:09.839061+0000","last_became_active":"2024-05-08T22:35:09.836629+0000","last_became_peered":"2024-05-08T22:35:09.836629+0000","last_unstale":"2024-05-08T22:35:09.839061+0000","last_undegraded":"2024-05-08T22:35:09.839061+0000","last_fullsized":"2024-05-08T22:35:09.839061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:43.475121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.841544+0000","last_change":"2024-05-08T22:35:09.837685+0000","last_active":"2024-05-08T22:35:11.841544+0000","last_peered":"2024-05-08T22:35:11.841544+0000","last_clean":"2024-05-08T22:35:11.841544+0000","last_became_active":"2024-05-08T22:35:09.837409+0000","last_became_peered":"2024-05-08T22:35:09.837409+0000","last_unstale":"2024-05-08T22:35:11.841544+0000","last_undegraded":"2024-05-08T22:35:11.841544+0000","last_fullsized":"2024-05-08T22:35:11.841544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:32.076361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825653+0000","last_change":"2024-05-08T22:35:09.833099+0000","last_active":"2024-05-08T22:35:11.825653+0000","last_peered":"2024-05-08T22:35:11.825653+0000","last_clean":"2024-05-08T22:35:11.825653+0000","last_became_active":"2024-05-08T22:35:09.832814+0000","last_became_peered":"2024-05-08T22:35:09.832814+0000","last_unstale":"2024-05-08T22:35:11.825653+0000","last_undegraded":"2024-05-08T22:35:11.825653+0000","last_fullsized":"2024-05-08T22:35:11.825653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:40.116060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812821+0000","last_change":"2024-05-08T22:35:09.818507+0000","last_active":"2024-05-08T22:35:10.812821+0000","last_peered":"2024-05-08T22:35:10.812821+0000","last_clean":"2024-05-08T22:35:10.812821+0000","last_became_active":"2024-05-08T22:35:09.818139+0000","last_became_peered":"2024-05-08T22:35:09.818139+0000","last_unstale":"2024-05-08T22:35:10.812821+0000","last_undegraded":"2024-05-08T22:35:10.812821+0000","last_fullsized":"2024-05-08T22:35:10.812821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:33.747011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:11.825503+0000","last_change":"2024-05-08T22:35:09.832232+0000","last_active":"2024-05-08T22:35:11.825503+0000","last_peered":"2024-05-08T22:35:11.825503+0000","last_clean":"2024-05-08T22:35:11.825503+0000","last_became_active":"2024-05-08T22:35:09.831889+0000","last_became_peered":"2024-05-08T22:35:09.831889+0000","last_unstale":"2024-05-08T22:35:11.825503+0000","last_undegraded":"2024-05-08T22:35:11.825503+0000","last_fullsized":"2024-05-08T22:35:11.825503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:01.802009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.812645+0000","last_change":"2024-05-08T22:35:09.838051+0000","last_active":"2024-05-08T22:35:10.812645+0000","last_peered":"2024-05-08T22:35:10.812645+0000","last_clean":"2024-05-08T22:35:10.812645+0000","last_became_active":"2024-05-08T22:35:09.837429+0000","last_became_peered":"2024-05-08T22:35:09.837429+0000","last_unstale":"2024-05-08T22:35:10.812645+0000","last_undegraded":"2024-05-08T22:35:10.812645+0000","last_fullsized":"2024-05-08T22:35:10.812645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.191374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:10.470060+0000","last_change":"2024-05-08T22:35:09.830834+0000","last_active":"2024-05-08T22:35:10.470060+0000","last_peered":"2024-05-08T22:35:10.470060+0000","last_clean":"2024-05-08T22:35:10.470060+0000","last_became_active":"2024-05-08T22:35:09.830475+0000","last_became_peered":"2024-05-08T22:35:09.830475+0000","last_unstale":"2024-05-08T22:35:10.470060+0000","last_undegraded":"2024-05-08T22:35:10.470060+0000","last_fullsized":"2024-05-08T22:35:10.470060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:06:10.996668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-08T22:35:09.838544+0000","last_change":"2024-05-08T22:35:09.836496+0000","last_active":"2024-05-08T22:35:09.838544+0000","last_peered":"2024-05-08T22:35:09.838544+0000","last_clean":"2024-05-08T22:35:09.838544+0000","last_became_active":"2024-05-08T22:35:09.835787+0000","last_became_peered":"2024-05-08T22:35:09.835787+0000","last_unstale":"2024-05-08T22:35:09.838544+0000","last_undegraded":"2024-05-08T22:35:09.838544+0000","last_fullsized":"2024-05-08T22:35:09.838544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T22:35:09.366728+0000","last_clean_scrub_stamp":"2024-05-08T22:35:09.366728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:30.395286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66435,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":348,"num_read_kb":248,"num_write":230,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":864256,"data_stored":410583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70599999999999996}]},{"osd":2,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.629},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.69499999999999995}]},{"osd":7,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78400000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":9,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.91800000000000004}]},{"osd":10,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.83099999999999996}]}]},{"osd":10,"up_from":67,"seq":287762808854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32636,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712516,"statfs":{"total":95995035648,"available":95961616384,"internally_reserved":0,"allocated":1560576,"data_stored":1061057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.44}]},{"osd":2,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.70099999999999996}]},{"osd":3,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55800000000000005}]},{"osd":4,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.78100000000000003}]},{"osd":9,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.72699999999999998}]},{"osd":11,"last update":"Wed May 8 22:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.81200000000000006}]}]},{"osd":9,"up_from":62,"seq":266287972377,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":888832,"data_stored":410561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.314},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.40899999999999997}]},{"osd":3,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":1.893},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.34399999999999997}]},{"osd":4,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.55200000000000005}]},{"osd":5,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.754},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":1.8300000000000001}]},{"osd":7,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.65900000000000003}]},{"osd":8,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.63},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.798}]},{"osd":10,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.8620000000000001}]}]},{"osd":8,"up_from":55,"seq":236223201308,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":880640,"data_stored":410658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65800000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.51800000000000002}]},{"osd":1,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80300000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81100000000000005,"5min":1.014,"15min":1.014},"last":0.72999999999999998}]},{"osd":2,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.435,"15min":0.433},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65100000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.501,"5min":0.47299999999999998,"15min":0.46800000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85399999999999998,"15min":0.85299999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.4630000000000001,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.35099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85099999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83699999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.623}]},{"osd":5,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76300000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72499999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54600000000000004}]},{"osd":6,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.95499999999999996,"15min":1.0089999999999999},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92800000000000005,"5min":7.3689999999999998,"15min":7.3689999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.90100000000000002,"15min":0.91900000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.202,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.82899999999999996}]},{"osd":7,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88600000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.81499999999999995}]},{"osd":9,"last update":"Wed May 8 22:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.749},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.79900000000000004}]},{"osd":10,"last update":"Wed May 8 22:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.88300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]}]},{"osd":7,"up_from":48,"seq":206158430240,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32556,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712596,"statfs":{"total":95995035648,"available":95961698304,"internally_reserved":0,"allocated":1478656,"data_stored":1003377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40100000000000002}]},{"osd":1,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26500000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.46899999999999997},"min":{"1min":0.33200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.496}]},{"osd":2,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.373,"5min":0.222,"15min":0.222},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.48199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.627,"15min":0.58699999999999997},"min":{"1min":0.39200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.4809999999999999,"5min":5.4809999999999999,"15min":5.4809999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.35499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.755},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76900000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.66600000000000004}]},{"osd":5,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.66700000000000004,"15min":0.64600000000000002},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.7229999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.48799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":0.85599999999999998}]},{"osd":6,"last update":"Wed May 8 22:35:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.76700000000000002,"15min":0.71899999999999997},"min":{"1min":0.54200000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.5650000000000004,"5min":6.5650000000000004,"15min":6.5650000000000004},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.72399999999999998,"15min":0.70799999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.2450000000000001,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":0.58399999999999996}]},{"osd":8,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.73199999999999998,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.0609999999999999,"5min":3.0609999999999999,"15min":3.0609999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.499,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90000000000000002}]},{"osd":9,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68200000000000005}]},{"osd":10,"last update":"Wed May 8 22:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.874},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.74399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":6,"up_from":43,"seq":184683593763,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":410230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39000000000000001,"15min":0.38800000000000001},"min":{"1min":0.29399999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.316},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42499999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.28999999999999998}]},{"osd":1,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.82399999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77000000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.44600000000000001}]},{"osd":3,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47699999999999998,"15min":0.48599999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56999999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.376},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.49299999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.35499999999999998}]},{"osd":4,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63},"min":{"1min":0.44800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77500000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.874,"15min":0.874},"last":0.499}]},{"osd":5,"last update":"Wed May 8 22:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.48199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.39500000000000002}]},{"osd":7,"last update":"Wed May 8 22:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92700000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.79700000000000004,"5min":0.874,"15min":0.874},"last":0.56699999999999995}]},{"osd":8,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94499999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71499999999999997,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.0149999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.48399999999999999}]},{"osd":9,"last update":"Wed May 8 22:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.52100000000000002}]},{"osd":10,"last update":"Wed May 8 22:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.69999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":929792,"data_stored":470289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48399999999999999,"15min":0.47199999999999998},"min":{"1min":0.317,"5min":0.311,"15min":0.311},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.435}]},{"osd":2,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83599999999999997,"5min":1.627,"15min":1.627},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.749}]},{"osd":3,"last update":"Wed May 8 22:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.35299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.40600000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.89300000000000002,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed May 8 22:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.44500000000000001,"15min":0.42499999999999999},"min":{"1min":0.33700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.47799999999999998,"15min":0.44400000000000001},"min":{"1min":0.33100000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.497}]},{"osd":5,"last update":"Wed May 8 22:34:52 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.80700000000000005}]},{"osd":6,"last update":"Wed May 8 22:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.47599999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed May 8 22:35:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.871},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.89700000000000002}]},{"osd":8,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96899999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.91400000000000003}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72399999999999998}]},{"osd":10,"last update":"Wed May 8 22:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.96599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":411569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 22:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.36399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.83699999999999997,"5min":0.996,"15min":0.996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.502},"min":{"1min":0.33500000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.74099999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed May 8 22:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.40200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51400000000000001,"15min":0.48799999999999999},"min":{"1min":0.36099999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.76500000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed May 8 22:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.39600000000000002,"15min":0.38700000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.42699999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 22:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56100000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.78700000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.53600000000000003,"15min":0.52900000000000003},"min":{"1min":0.38200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.69599999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48299999999999998}]},{"osd":5,"last update":"Wed May 8 22:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70599999999999996,"5min":0.751,"15min":0.751},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90000000000000002,"5min":1.085,"15min":1.085},"last":0.624}]},{"osd":6,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78700000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83899999999999997,"5min":0.872,"15min":0.872},"last":0.5}]},{"osd":7,"last update":"Wed May 8 22:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53500000000000003}]},{"osd":8,"last update":"Wed May 8 22:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.79900000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.745,"15min":0.76500000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.81200000000000006,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.63900000000000001}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.58899999999999997}]},{"osd":10,"last update":"Wed May 8 22:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.54000000000000004}]},{"osd":11,"last update":"Wed May 8 22:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65900000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":950272,"data_stored":470458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71799999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48599999999999999}]},{"osd":1,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.40500000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68200000000000005}]},{"osd":3,"last update":"Wed May 8 22:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69199999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.50800000000000001,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed May 8 22:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59599999999999997}]},{"osd":5,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.33200000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64100000000000001,"5min":0.748,"15min":0.748},"last":0.38700000000000001}]},{"osd":6,"last update":"Wed May 8 22:35:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.50600000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64400000000000002}]},{"osd":7,"last update":"Wed May 8 22:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.47299999999999998}]},{"osd":8,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65400000000000003}]},{"osd":9,"last update":"Wed May 8 22:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70399999999999996}]},{"osd":10,"last update":"Wed May 8 22:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":3,"up_from":24,"seq":103079215150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36076,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709076,"statfs":{"total":95995035648,"available":95958093824,"internally_reserved":0,"allocated":888832,"data_stored":410103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.33600000000000002},"min":{"1min":0.27200000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.54100000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.36399999999999999,"15min":0.35199999999999998},"min":{"1min":0.28899999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58899999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.373}]},{"osd":1,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72999999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54100000000000004,"15min":0.52800000000000002},"min":{"1min":0.38800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Wed May 8 22:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88100000000000001,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed May 8 22:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56200000000000006,"15min":0.53800000000000003},"min":{"1min":0.44800000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78600000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80600000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.52000000000000002}]},{"osd":5,"last update":"Wed May 8 22:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85499999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed May 8 22:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.41399999999999998}]},{"osd":7,"last update":"Wed May 8 22:35:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.63200000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73299999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.47899999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91400000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.59399999999999997}]},{"osd":8,"last update":"Wed May 8 22:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.496},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.53900000000000003}]},{"osd":9,"last update":"Wed May 8 22:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.504}]},{"osd":10,"last update":"Wed May 8 22:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.629}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":872448,"data_stored":410484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.38100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.42699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40100000000000002},"min":{"1min":0.29899999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64800000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.504},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.41599999999999998},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.64500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.40600000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58399999999999996,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66600000000000004}]},{"osd":3,"last update":"Wed May 8 22:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.34499999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66600000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53300000000000003,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.318,"15min":0.318},"max":{"1min":0.64700000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56000000000000005}]},{"osd":5,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89200000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.89000000000000001}]},{"osd":6,"last update":"Wed May 8 22:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.40699999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed May 8 22:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94599999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.75700000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61699999999999999}]},{"osd":8,"last update":"Wed May 8 22:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78700000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.93300000000000005}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.90500000000000003}]},{"osd":10,"last update":"Wed May 8 22:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.94299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":5,"up_from":36,"seq":154618822695,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1490944,"data_stored":1002758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.50800000000000001,"15min":0.499},"min":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.35899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.85299999999999998}]},{"osd":1,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.82299999999999995}]},{"osd":2,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.379,"15min":0.372},"min":{"1min":0.28199999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.503}]},{"osd":3,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65300000000000002}]},{"osd":4,"last update":"Wed May 8 22:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.755},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.88500000000000001}]},{"osd":6,"last update":"Wed May 8 22:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40899999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72499999999999998}]},{"osd":7,"last update":"Wed May 8 22:35:27 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81499999999999995,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86899999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Wed May 8 22:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70999999999999996,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55600000000000005,"15min":0.56100000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.629}]},{"osd":9,"last update":"Wed May 8 22:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80400000000000005}]},{"osd":10,"last update":"Wed May 8 22:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T22:35:49.145 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T22:35:49.145 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T22:35:49.145 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-08T22:35:49.146 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-08T22:35:49.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:48 smithi066 ceph-mon[46830]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:49.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:48 smithi079 ceph-mon[46738]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:50.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:49 smithi062 ceph-mon[42571]: pgmap v241: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-08T22:35:50.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:49 smithi062 ceph-mon[42571]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.287+0000 7f6503b6a700 1 -- 172.21.15.62:0/1745660060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc100890 msgr2=0x7f64fc10cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.287+0000 7f6503b6a700 1 --2- 172.21.15.62:0/1745660060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc100890 0x7f64fc10cd40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f64f400b0a0 tx=0x7f64f400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.287+0000 7f6503b6a700 1 -- 172.21.15.62:0/1745660060 shutdown_connections 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.287+0000 7f6503b6a700 1 --2- 172.21.15.62:0/1745660060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc100890 0x7f64fc10cd40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.287+0000 7f6503b6a700 1 --2- 172.21.15.62:0/1745660060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64fc105f00 0x7f64fc1062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.287+0000 7f6503b6a700 1 --2- 172.21.15.62:0/1745660060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64fc0ffed0 0x7f64fc100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.287+0000 7f6503b6a700 1 -- 172.21.15.62:0/1745660060 >> 172.21.15.62:0/1745660060 conn(0x7f64fc075f80 msgr2=0x7f64fc076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.287+0000 7f6503b6a700 1 -- 172.21.15.62:0/1745660060 shutdown_connections 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.287+0000 7f6503b6a700 1 -- 172.21.15.62:0/1745660060 wait complete. 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6503b6a700 1 Processor -- start 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6503b6a700 1 -- start start 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6503b6a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc0ffed0 0x7f64fc101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6503b6a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64fc100890 0x7f64fc1020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6503b6a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64fc105f00 0x7f64fc1025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6503b6a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1a5990 con 0x7f64fc0ffed0 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6503b6a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1a5b00 con 0x7f64fc100890 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6503b6a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1a5cc0 con 0x7f64fc105f00 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6501906700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc0ffed0 0x7f64fc101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6501906700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc0ffed0 0x7f64fc101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44052/0 (socket says 172.21.15.62:44052) 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6501906700 1 -- 172.21.15.62:0/1123666135 learned_addr learned my addr 172.21.15.62:0/1123666135 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6501906700 1 -- 172.21.15.62:0/1123666135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64fc105f00 msgr2=0x7f64fc1025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6501906700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64fc105f00 0x7f64fc1025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6501906700 1 -- 172.21.15.62:0/1123666135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64fc100890 msgr2=0x7f64fc1020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6501105700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64fc100890 0x7f64fc1020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6501906700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64fc100890 0x7f64fc1020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6501906700 1 -- 172.21.15.62:0/1123666135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f4009d00 con 0x7f64fc0ffed0 2024-05-08T22:35:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.288+0000 7f6501105700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64fc100890 0x7f64fc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.289+0000 7f6501906700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc0ffed0 0x7f64fc101b70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64f800d950 tx=0x7f64f800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.289+0000 7f64f2ffd700 1 -- 172.21.15.62:0/1123666135 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8009b40 con 0x7f64fc0ffed0 2024-05-08T22:35:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.289+0000 7f64f2ffd700 1 -- 172.21.15.62:0/1123666135 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64f8009ca0 con 0x7f64fc0ffed0 2024-05-08T22:35:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.289+0000 7f6503b6a700 1 -- 172.21.15.62:0/1123666135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc1a5fa0 con 0x7f64fc0ffed0 2024-05-08T22:35:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.289+0000 7f64f2ffd700 1 -- 172.21.15.62:0/1123666135 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f800f620 con 0x7f64fc0ffed0 2024-05-08T22:35:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.289+0000 7f6503b6a700 1 -- 172.21.15.62:0/1123666135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc1a6410 con 0x7f64fc0ffed0 2024-05-08T22:35:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.290+0000 7f6503b6a700 1 -- 172.21.15.62:0/1123666135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc1015b0 con 0x7f64fc0ffed0 2024-05-08T22:35:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.291+0000 7f64f2ffd700 1 -- 172.21.15.62:0/1123666135 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f64f8010460 con 0x7f64fc0ffed0 2024-05-08T22:35:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.291+0000 7f64f2ffd700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64e80a11b0 0x7f64e80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.292+0000 7f6501105700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64e80a11b0 0x7f64e80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.292+0000 7f64f2ffd700 1 -- 172.21.15.62:0/1123666135 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f64f80ccc30 con 0x7f64fc0ffed0 2024-05-08T22:35:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.292+0000 7f6501105700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64e80a11b0 0x7f64e80a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f64fc1032a0 tx=0x7f64ec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:50.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.296+0000 7f64f2ffd700 1 -- 172.21.15.62:0/1123666135 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f64f8096410 con 0x7f64fc0ffed0 2024-05-08T22:35:50.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:49 smithi066 ceph-mon[46830]: pgmap v241: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-08T22:35:50.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:49 smithi066 ceph-mon[46830]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:50.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:49 smithi079 ceph-mon[46738]: pgmap v241: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-08T22:35:50.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:49 smithi079 ceph-mon[46738]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T22:35:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.544+0000 7f6503b6a700 1 -- 172.21.15.62:0/1123666135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f64fc04ec60 con 0x7f64fc0ffed0 2024-05-08T22:35:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.544+0000 7f64f2ffd700 1 -- 172.21.15.62:0/1123666135 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f64f8020040 con 0x7f64fc0ffed0 2024-05-08T22:35:50.545 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:50.545 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 -- 172.21.15.62:0/1123666135 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64e80a11b0 msgr2=0x7f64e80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64e80a11b0 0x7f64e80a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f64fc1032a0 tx=0x7f64ec008040 comp rx=0 tx=0).stop 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 -- 172.21.15.62:0/1123666135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc0ffed0 msgr2=0x7f64fc101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc0ffed0 0x7f64fc101b70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64f800d950 tx=0x7f64f800dd10 comp rx=0 tx=0).stop 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 -- 172.21.15.62:0/1123666135 shutdown_connections 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64fc105f00 0x7f64fc1025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc0ffed0 0x7f64fc101b70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64e80a11b0 0x7f64e80a3670 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 --2- 172.21.15.62:0/1123666135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64fc100890 0x7f64fc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 -- 172.21.15.62:0/1123666135 >> 172.21.15.62:0/1123666135 conn(0x7f64fc075f80 msgr2=0x7f64fc068af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 -- 172.21.15.62:0/1123666135 shutdown_connections 2024-05-08T22:35:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:50.547+0000 7f6503b6a700 1 -- 172.21.15.62:0/1123666135 wait complete. 2024-05-08T22:35:50.906 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-08T22:35:50.906 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:51.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.038+0000 7fc3dabd8700 1 -- 172.21.15.62:0/320196246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d4101910 msgr2=0x7fc3d41081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.038+0000 7fc3dabd8700 1 --2- 172.21.15.62:0/320196246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d4101910 0x7fc3d41081c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00b0d0 tx=0x7fc3cc00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.038+0000 7fc3dabd8700 1 -- 172.21.15.62:0/320196246 shutdown_connections 2024-05-08T22:35:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.038+0000 7fc3dabd8700 1 --2- 172.21.15.62:0/320196246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d4101910 0x7fc3d41081c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.038+0000 7fc3dabd8700 1 --2- 172.21.15.62:0/320196246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3d40ffd80 0x7fc3d41001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.038+0000 7fc3dabd8700 1 --2- 172.21.15.62:0/320196246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3d4100f70 0x7fc3d41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.038+0000 7fc3dabd8700 1 -- 172.21.15.62:0/320196246 >> 172.21.15.62:0/320196246 conn(0x7fc3d40fb310 msgr2=0x7fc3d40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.038+0000 7fc3dabd8700 1 -- 172.21.15.62:0/320196246 shutdown_connections 2024-05-08T22:35:51.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.038+0000 7fc3dabd8700 1 -- 172.21.15.62:0/320196246 wait complete. 2024-05-08T22:35:51.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3dabd8700 1 Processor -- start 2024-05-08T22:35:51.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3dabd8700 1 -- start start 2024-05-08T22:35:51.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3dabd8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d40ffd80 0x7fc3d4195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3dabd8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3d4100f70 0x7fc3d41962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3dabd8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3d4101910 0x7fc3d419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3dabd8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419d8d0 con 0x7fc3d40ffd80 2024-05-08T22:35:51.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3dabd8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419da10 con 0x7fc3d4100f70 2024-05-08T22:35:51.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3dabd8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419dba0 con 0x7fc3d4101910 2024-05-08T22:35:51.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3d8974700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d40ffd80 0x7fc3d4195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3d8974700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d40ffd80 0x7fc3d4195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44062/0 (socket says 172.21.15.62:44062) 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3d8974700 1 -- 172.21.15.62:0/1721424897 learned_addr learned my addr 172.21.15.62:0/1721424897 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3d9175700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3d4101910 0x7fc3d419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.039+0000 7fc3d3fff700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3d4100f70 0x7fc3d41962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3d8974700 1 -- 172.21.15.62:0/1721424897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3d4101910 msgr2=0x7fc3d419d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3d8974700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3d4101910 0x7fc3d419d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3d8974700 1 -- 172.21.15.62:0/1721424897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3d4100f70 msgr2=0x7fc3d41962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3d8974700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3d4100f70 0x7fc3d41962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3d8974700 1 -- 172.21.15.62:0/1721424897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc009d00 con 0x7fc3d40ffd80 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3d3fff700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3d4100f70 0x7fc3d41962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3d8974700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d40ffd80 0x7fc3d4195da0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800c990 tx=0x7fc3c800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3d1ffb700 1 -- 172.21.15.62:0/1721424897 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8007b00 con 0x7fc3d40ffd80 2024-05-08T22:35:51.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3d1ffb700 1 -- 172.21.15.62:0/1721424897 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3c8007c60 con 0x7fc3d40ffd80 2024-05-08T22:35:51.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3dabd8700 1 -- 172.21.15.62:0/1721424897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d419de80 con 0x7fc3d40ffd80 2024-05-08T22:35:51.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3d1ffb700 1 -- 172.21.15.62:0/1721424897 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8018660 con 0x7fc3d40ffd80 2024-05-08T22:35:51.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.040+0000 7fc3dabd8700 1 -- 172.21.15.62:0/1721424897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d419e2b0 con 0x7fc3d40ffd80 2024-05-08T22:35:51.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.041+0000 7fc3dabd8700 1 -- 172.21.15.62:0/1721424897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d41070d0 con 0x7fc3d40ffd80 2024-05-08T22:35:51.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.042+0000 7fc3d1ffb700 1 -- 172.21.15.62:0/1721424897 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc3c800f450 con 0x7fc3d40ffd80 2024-05-08T22:35:51.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.043+0000 7fc3d1ffb700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3c40a1230 0x7fc3c40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.043+0000 7fc3d3fff700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3c40a1230 0x7fc3c40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.043+0000 7fc3d1ffb700 1 -- 172.21.15.62:0/1721424897 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3c80cc8c0 con 0x7fc3d40ffd80 2024-05-08T22:35:51.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.044+0000 7fc3d3fff700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3c40a1230 0x7fc3c40a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0005950 tx=0x7fc3c00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:51.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.048+0000 7fc3d1ffb700 1 -- 172.21.15.62:0/1721424897 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc3c8095fa0 con 0x7fc3d40ffd80 2024-05-08T22:35:51.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:50 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1123666135' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T22:35:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.257+0000 7fc3dabd8700 1 -- 172.21.15.62:0/1721424897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3d410b620 con 0x7fc3d40ffd80 2024-05-08T22:35:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.259+0000 7fc3d1ffb700 1 -- 172.21.15.62:0/1721424897 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22047 (secure 0 0 0) 0x7fc3c80995c0 con 0x7fc3d40ffd80 2024-05-08T22:35:51.259 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:51.259 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":79,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:35:10.805199+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 -- 172.21.15.62:0/1721424897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3c40a1230 msgr2=0x7fc3c40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3c40a1230 0x7fc3c40a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0005950 tx=0x7fc3c00058e0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 -- 172.21.15.62:0/1721424897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d40ffd80 msgr2=0x7fc3d4195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d40ffd80 0x7fc3d4195da0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800c990 tx=0x7fc3c800cca0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 -- 172.21.15.62:0/1721424897 shutdown_connections 2024-05-08T22:35:51.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3d4101910 0x7fc3d419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3d40ffd80 0x7fc3d4195da0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3c40a1230 0x7fc3c40a36f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 --2- 172.21.15.62:0/1721424897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3d4100f70 0x7fc3d41962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 -- 172.21.15.62:0/1721424897 >> 172.21.15.62:0/1721424897 conn(0x7fc3d40fb310 msgr2=0x7fc3d4104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:51.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 -- 172.21.15.62:0/1721424897 shutdown_connections 2024-05-08T22:35:51.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.261+0000 7fc3dabd8700 1 -- 172.21.15.62:0/1721424897 wait complete. 2024-05-08T22:35:51.273 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:31:54.466249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:35:08.296334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:35:09.167901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:35:51.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:35:51.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:50 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1123666135' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T22:35:51.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:50 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1123666135' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T22:35:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.405+0000 7fd158b70700 1 -- 172.21.15.62:0/1107361080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd154100e70 msgr2=0x7fd154101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.405+0000 7fd158b70700 1 --2- 172.21.15.62:0/1107361080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd154100e70 0x7fd154101290 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd148009990 tx=0x7fd148009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.405+0000 7fd158b70700 1 -- 172.21.15.62:0/1107361080 shutdown_connections 2024-05-08T22:35:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.405+0000 7fd158b70700 1 --2- 172.21.15.62:0/1107361080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd154102980 0x7fd1541092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.405+0000 7fd158b70700 1 --2- 172.21.15.62:0/1107361080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd154100e70 0x7fd154101290 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.405+0000 7fd158b70700 1 --2- 172.21.15.62:0/1107361080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd154101fc0 0x7fd154102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.405+0000 7fd158b70700 1 -- 172.21.15.62:0/1107361080 >> 172.21.15.62:0/1107361080 conn(0x7fd1540fc400 msgr2=0x7fd1540fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:51.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.405+0000 7fd158b70700 1 -- 172.21.15.62:0/1107361080 shutdown_connections 2024-05-08T22:35:51.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.405+0000 7fd158b70700 1 -- 172.21.15.62:0/1107361080 wait complete. 2024-05-08T22:35:51.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd158b70700 1 Processor -- start 2024-05-08T22:35:51.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd158b70700 1 -- start start 2024-05-08T22:35:51.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd158b70700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd154100e70 0x7fd154074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd158b70700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd154101fc0 0x7fd154073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd158b70700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd154102980 0x7fd1540738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd158b70700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd154074070 con 0x7fd154102980 2024-05-08T22:35:51.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd158b70700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540741e0 con 0x7fd154100e70 2024-05-08T22:35:51.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd158b70700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1541a0e90 con 0x7fd154101fc0 2024-05-08T22:35:51.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd152d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd154102980 0x7fd1540738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd152d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd154102980 0x7fd1540738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44080/0 (socket says 172.21.15.62:44080) 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd152d9d700 1 -- 172.21.15.62:0/2423283524 learned_addr learned my addr 172.21.15.62:0/2423283524 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd151d9b700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd154101fc0 0x7fd154073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd15259c700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd154100e70 0x7fd154074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd152d9d700 1 -- 172.21.15.62:0/2423283524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd154101fc0 msgr2=0x7fd154073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd152d9d700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd154101fc0 0x7fd154073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd152d9d700 1 -- 172.21.15.62:0/2423283524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd154100e70 msgr2=0x7fd154074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd152d9d700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd154100e70 0x7fd154074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd152d9d700 1 -- 172.21.15.62:0/2423283524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480095f0 con 0x7fd154102980 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.406+0000 7fd152d9d700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd154102980 0x7fd1540738d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd14c007ee0 tx=0x7fd14c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.407+0000 7fd1437fe700 1 -- 172.21.15.62:0/2423283524 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00dd70 con 0x7fd154102980 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.407+0000 7fd158b70700 1 -- 172.21.15.62:0/2423283524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1541a1120 con 0x7fd154102980 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.407+0000 7fd1437fe700 1 -- 172.21.15.62:0/2423283524 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd14c00f040 con 0x7fd154102980 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.407+0000 7fd1437fe700 1 -- 172.21.15.62:0/2423283524 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0048a0 con 0x7fd154102980 2024-05-08T22:35:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.407+0000 7fd158b70700 1 -- 172.21.15.62:0/2423283524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1541a1630 con 0x7fd154102980 2024-05-08T22:35:51.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.408+0000 7fd158b70700 1 -- 172.21.15.62:0/2423283524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1541082e0 con 0x7fd154102980 2024-05-08T22:35:51.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.409+0000 7fd1437fe700 1 -- 172.21.15.62:0/2423283524 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd14c003a00 con 0x7fd154102980 2024-05-08T22:35:51.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.409+0000 7fd1437fe700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd13c0a11b0 0x7fd13c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.409+0000 7fd1437fe700 1 -- 172.21.15.62:0/2423283524 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd14c0cc1d0 con 0x7fd154102980 2024-05-08T22:35:51.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.409+0000 7fd15259c700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd13c0a11b0 0x7fd13c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.410+0000 7fd15259c700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd13c0a11b0 0x7fd13c0a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1480098e0 tx=0x7fd148000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:51.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.414+0000 7fd1437fe700 1 -- 172.21.15.62:0/2423283524 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd14c0959b0 con 0x7fd154102980 2024-05-08T22:35:51.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.621+0000 7fd158b70700 1 -- 172.21.15.62:0/2423283524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd15410c6c0 con 0x7fd154102980 2024-05-08T22:35:51.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.621+0000 7fd1437fe700 1 -- 172.21.15.62:0/2423283524 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd14c098fd0 con 0x7fd154102980 2024-05-08T22:35:51.621 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-05-08T22:35:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.623+0000 7fd158b70700 1 -- 172.21.15.62:0/2423283524 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd13c0a11b0 msgr2=0x7fd13c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.623+0000 7fd158b70700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd13c0a11b0 0x7fd13c0a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1480098e0 tx=0x7fd148000bc0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.623+0000 7fd158b70700 1 -- 172.21.15.62:0/2423283524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd154102980 msgr2=0x7fd1540738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.623+0000 7fd158b70700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd154102980 0x7fd1540738d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd14c007ee0 tx=0x7fd14c00d3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.624+0000 7fd158b70700 1 -- 172.21.15.62:0/2423283524 shutdown_connections 2024-05-08T22:35:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.624+0000 7fd158b70700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd154102980 0x7fd1540738d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.624+0000 7fd158b70700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd13c0a11b0 0x7fd13c0a3670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.624+0000 7fd158b70700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd154100e70 0x7fd154074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.624+0000 7fd158b70700 1 --2- 172.21.15.62:0/2423283524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd154101fc0 0x7fd154073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.624+0000 7fd158b70700 1 -- 172.21.15.62:0/2423283524 >> 172.21.15.62:0/2423283524 conn(0x7fd1540fc400 msgr2=0x7fd154105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.624+0000 7fd158b70700 1 -- 172.21.15.62:0/2423283524 shutdown_connections 2024-05-08T22:35:51.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.624+0000 7fd158b70700 1 -- 172.21.15.62:0/2423283524 wait complete. 2024-05-08T22:35:51.636 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:35:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.766+0000 7fed6f716700 1 -- 172.21.15.62:0/1795094224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed68101fc0 msgr2=0x7fed68102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.766+0000 7fed6f716700 1 --2- 172.21.15.62:0/1795094224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed68101fc0 0x7fed68102440 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fed64009990 tx=0x7fed64009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.766+0000 7fed6f716700 1 -- 172.21.15.62:0/1795094224 shutdown_connections 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.766+0000 7fed6f716700 1 --2- 172.21.15.62:0/1795094224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed68102980 0x7fed681092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.766+0000 7fed6f716700 1 --2- 172.21.15.62:0/1795094224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed68100e70 0x7fed68101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.766+0000 7fed6f716700 1 --2- 172.21.15.62:0/1795094224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed68101fc0 0x7fed68102440 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.766+0000 7fed6f716700 1 -- 172.21.15.62:0/1795094224 >> 172.21.15.62:0/1795094224 conn(0x7fed680fc400 msgr2=0x7fed680fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.766+0000 7fed6f716700 1 -- 172.21.15.62:0/1795094224 shutdown_connections 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.766+0000 7fed6f716700 1 -- 172.21.15.62:0/1795094224 wait complete. 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.766+0000 7fed6f716700 1 Processor -- start 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.766+0000 7fed6f716700 1 -- start start 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6f716700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed68100e70 0x7fed68196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6f716700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed68101fc0 0x7fed681972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6f716700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed68102980 0x7fed6819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6f716700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6819e8d0 con 0x7fed68100e70 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6f716700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6819ea10 con 0x7fed68102980 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6f716700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6819eba0 con 0x7fed68101fc0 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6d4b2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed68100e70 0x7fed68196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6d4b2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed68100e70 0x7fed68196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44090/0 (socket says 172.21.15.62:44090) 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6d4b2700 1 -- 172.21.15.62:0/335609501 learned_addr learned my addr 172.21.15.62:0/335609501 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6ccb1700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed68101fc0 0x7fed681972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6dcb3700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed68102980 0x7fed6819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6d4b2700 1 -- 172.21.15.62:0/335609501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed68101fc0 msgr2=0x7fed681972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6d4b2700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed68101fc0 0x7fed681972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6d4b2700 1 -- 172.21.15.62:0/335609501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed68102980 msgr2=0x7fed6819e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6d4b2700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed68102980 0x7fed6819e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6d4b2700 1 -- 172.21.15.62:0/335609501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640095f0 con 0x7fed68100e70 2024-05-08T22:35:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6d4b2700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed68100e70 0x7fed68196da0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fed5800e9a0 tx=0x7fed5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed5e7fc700 1 -- 172.21.15.62:0/335609501 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58009790 con 0x7fed68100e70 2024-05-08T22:35:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed5e7fc700 1 -- 172.21.15.62:0/335609501 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed58010430 con 0x7fed68100e70 2024-05-08T22:35:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6f716700 1 -- 172.21.15.62:0/335609501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6819ee80 con 0x7fed68100e70 2024-05-08T22:35:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed5e7fc700 1 -- 172.21.15.62:0/335609501 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58018680 con 0x7fed68100e70 2024-05-08T22:35:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.767+0000 7fed6f716700 1 -- 172.21.15.62:0/335609501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6819f2b0 con 0x7fed68100e70 2024-05-08T22:35:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.768+0000 7fed6f716700 1 -- 172.21.15.62:0/335609501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed68108230 con 0x7fed68100e70 2024-05-08T22:35:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.769+0000 7fed5e7fc700 1 -- 172.21.15.62:0/335609501 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fed580105a0 con 0x7fed68100e70 2024-05-08T22:35:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.770+0000 7fed5e7fc700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fed540a1230 0x7fed540a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.770+0000 7fed5e7fc700 1 -- 172.21.15.62:0/335609501 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fed58014070 con 0x7fed68100e70 2024-05-08T22:35:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.770+0000 7fed6ccb1700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fed540a1230 0x7fed540a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.770+0000 7fed6ccb1700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fed540a1230 0x7fed540a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b2e0 tx=0x7fed6400b6f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:51.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.773+0000 7fed5e7fc700 1 -- 172.21.15.62:0/335609501 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fed58096400 con 0x7fed68100e70 2024-05-08T22:35:51.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.981+0000 7fed6f716700 1 -- 172.21.15.62:0/335609501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed6810c6f0 con 0x7fed68100e70 2024-05-08T22:35:51.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.981+0000 7fed5e7fc700 1 -- 172.21.15.62:0/335609501 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed580260a0 con 0x7fed68100e70 2024-05-08T22:35:51.981 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-05-08T22:35:51.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 -- 172.21.15.62:0/335609501 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fed540a1230 msgr2=0x7fed540a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fed540a1230 0x7fed540a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b2e0 tx=0x7fed6400b6f0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 -- 172.21.15.62:0/335609501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed68100e70 msgr2=0x7fed68196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:51.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed68100e70 0x7fed68196da0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fed5800e9a0 tx=0x7fed5800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 -- 172.21.15.62:0/335609501 shutdown_connections 2024-05-08T22:35:51.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fed540a1230 0x7fed540a36f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed68102980 0x7fed6819e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed68100e70 0x7fed68196da0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 --2- 172.21.15.62:0/335609501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed68101fc0 0x7fed681972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:51.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 -- 172.21.15.62:0/335609501 >> 172.21.15.62:0/335609501 conn(0x7fed680fc400 msgr2=0x7fed68105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:51.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 -- 172.21.15.62:0/335609501 shutdown_connections 2024-05-08T22:35:51.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:51.984+0000 7fed6f716700 1 -- 172.21.15.62:0/335609501 wait complete. 2024-05-08T22:35:51.996 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:35:52.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 -- 172.21.15.62:0/587119546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c100f70 msgr2=0x7f0d0c1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 --2- 172.21.15.62:0/587119546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c100f70 0x7f0d0c1013d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0099c0 tx=0x7f0cfc009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 -- 172.21.15.62:0/587119546 shutdown_connections 2024-05-08T22:35:52.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 --2- 172.21.15.62:0/587119546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0c101910 0x7f0d0c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 --2- 172.21.15.62:0/587119546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0c0ffd80 0x7f0d0c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 --2- 172.21.15.62:0/587119546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c100f70 0x7f0d0c1013d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 -- 172.21.15.62:0/587119546 >> 172.21.15.62:0/587119546 conn(0x7f0d0c0fb310 msgr2=0x7f0d0c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 -- 172.21.15.62:0/587119546 shutdown_connections 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 -- 172.21.15.62:0/587119546 wait complete. 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 Processor -- start 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 -- start start 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0c0ffd80 0x7f0d0c103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0c100f70 0x7f0d0c102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c101910 0x7f0d0c102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c103030 con 0x7f0d0c101910 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c1031a0 con 0x7f0d0c0ffd80 2024-05-08T22:35:52.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.127+0000 7f0d1376f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c1a0e90 con 0x7f0d0c100f70 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d11d0c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c101910 0x7f0d0c102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d11d0c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c101910 0x7f0d0c102890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44110/0 (socket says 172.21.15.62:44110) 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d11d0c700 1 -- 172.21.15.62:0/4127213897 learned_addr learned my addr 172.21.15.62:0/4127213897 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d1150b700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0c0ffd80 0x7f0d0c103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d10d0a700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0c100f70 0x7f0d0c102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d11d0c700 1 -- 172.21.15.62:0/4127213897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0c100f70 msgr2=0x7f0d0c102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d11d0c700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0c100f70 0x7f0d0c102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d11d0c700 1 -- 172.21.15.62:0/4127213897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0c0ffd80 msgr2=0x7f0d0c103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d11d0c700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0c0ffd80 0x7f0d0c103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d11d0c700 1 -- 172.21.15.62:0/4127213897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc0095f0 con 0x7f0d0c101910 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d11d0c700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c101910 0x7f0d0c102890 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400ef30 tx=0x7f0d0400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d027fc700 1 -- 172.21.15.62:0/4127213897 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04018070 con 0x7f0d0c101910 2024-05-08T22:35:52.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d1376f700 1 -- 172.21.15.62:0/4127213897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0c1a10c0 con 0x7f0d0c101910 2024-05-08T22:35:52.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d027fc700 1 -- 172.21.15.62:0/4127213897 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d0400f040 con 0x7f0d0c101910 2024-05-08T22:35:52.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d027fc700 1 -- 172.21.15.62:0/4127213897 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0400cd50 con 0x7f0d0c101910 2024-05-08T22:35:52.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.128+0000 7f0d1376f700 1 -- 172.21.15.62:0/4127213897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c1a1580 con 0x7f0d0c101910 2024-05-08T22:35:52.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.129+0000 7f0d1376f700 1 -- 172.21.15.62:0/4127213897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0c068160 con 0x7f0d0c101910 2024-05-08T22:35:52.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.131+0000 7f0d027fc700 1 -- 172.21.15.62:0/4127213897 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0d04003a00 con 0x7f0d0c101910 2024-05-08T22:35:52.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.132+0000 7f0d027fc700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cf80a11b0 0x7f0cf80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.132+0000 7f0d1150b700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cf80a11b0 0x7f0cf80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.132+0000 7f0d027fc700 1 -- 172.21.15.62:0/4127213897 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0d040ccf70 con 0x7f0d0c101910 2024-05-08T22:35:52.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.132+0000 7f0d1150b700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cf80a11b0 0x7f0cf80a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c100dd0 tx=0x7f0d0800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:52.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.135+0000 7f0d027fc700 1 -- 172.21.15.62:0/4127213897 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0d040966e0 con 0x7f0d0c101910 2024-05-08T22:35:52.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:51 smithi062 ceph-mon[42571]: pgmap v242: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-08T22:35:52.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:51 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1721424897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:52.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:51 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2423283524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:35:52.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:51 smithi066 ceph-mon[46830]: pgmap v242: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-08T22:35:52.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:51 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1721424897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:52.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:51 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2423283524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:35:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.341+0000 7f0d1376f700 1 -- 172.21.15.62:0/4127213897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d0c10b770 con 0x7f0d0c101910 2024-05-08T22:35:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.342+0000 7f0d027fc700 1 -- 172.21.15.62:0/4127213897 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d04099d00 con 0x7f0d0c101910 2024-05-08T22:35:52.342 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-05-08T22:35:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 -- 172.21.15.62:0/4127213897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cf80a11b0 msgr2=0x7f0cf80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cf80a11b0 0x7f0cf80a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c100dd0 tx=0x7f0d0800b410 comp rx=0 tx=0).stop 2024-05-08T22:35:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 -- 172.21.15.62:0/4127213897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c101910 msgr2=0x7f0d0c102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c101910 0x7f0d0c102890 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400ef30 tx=0x7f0d0400c3a0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 -- 172.21.15.62:0/4127213897 shutdown_connections 2024-05-08T22:35:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c101910 0x7f0d0c102890 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f0cf80a11b0 0x7f0cf80a3670 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0c0ffd80 0x7f0d0c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 --2- 172.21.15.62:0/4127213897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0c100f70 0x7f0d0c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 -- 172.21.15.62:0/4127213897 >> 172.21.15.62:0/4127213897 conn(0x7f0d0c0fb310 msgr2=0x7f0d0c0fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 -- 172.21.15.62:0/4127213897 shutdown_connections 2024-05-08T22:35:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.344+0000 7f0d1376f700 1 -- 172.21.15.62:0/4127213897 wait complete. 2024-05-08T22:35:52.357 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:52.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:51 smithi079 ceph-mon[46738]: pgmap v242: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-08T22:35:52.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:51 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1721424897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:52.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:51 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2423283524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:35:52.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.483+0000 7f41d3670700 1 -- 172.21.15.62:0/839985530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41cc0ff8d0 msgr2=0x7f41cc1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.483+0000 7f41d3670700 1 --2- 172.21.15.62:0/839985530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41cc0ff8d0 0x7f41cc1094a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41c400b0a0 tx=0x7f41c400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.483+0000 7f41d3670700 1 -- 172.21.15.62:0/839985530 shutdown_connections 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.483+0000 7f41d3670700 1 --2- 172.21.15.62:0/839985530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41cc0ff8d0 0x7f41cc1094a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.483+0000 7f41d3670700 1 --2- 172.21.15.62:0/839985530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41cc1024c0 0x7f41cc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.483+0000 7f41d3670700 1 --2- 172.21.15.62:0/839985530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc0fef30 0x7f41cc0ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.483+0000 7f41d3670700 1 -- 172.21.15.62:0/839985530 >> 172.21.15.62:0/839985530 conn(0x7f41cc0faa90 msgr2=0x7f41cc0fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.483+0000 7f41d3670700 1 -- 172.21.15.62:0/839985530 shutdown_connections 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.483+0000 7f41d3670700 1 -- 172.21.15.62:0/839985530 wait complete. 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d3670700 1 Processor -- start 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d3670700 1 -- start start 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d3670700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41cc0fef30 0x7f41cc196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d3670700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc0ff8d0 0x7f41cc1973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d3670700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41cc1024c0 0x7f41cc19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d3670700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc19e9e0 con 0x7f41cc0fef30 2024-05-08T22:35:52.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d3670700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc19eb20 con 0x7f41cc1024c0 2024-05-08T22:35:52.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d3670700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc19ecb0 con 0x7f41cc0ff8d0 2024-05-08T22:35:52.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d140c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41cc0fef30 0x7f41cc196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d140c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41cc0fef30 0x7f41cc196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44122/0 (socket says 172.21.15.62:44122) 2024-05-08T22:35:52.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d140c700 1 -- 172.21.15.62:0/3223767710 learned_addr learned my addr 172.21.15.62:0/3223767710 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:52.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d0c0b700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc0ff8d0 0x7f41cc1973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d1c0d700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41cc1024c0 0x7f41cc19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d140c700 1 -- 172.21.15.62:0/3223767710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc0ff8d0 msgr2=0x7f41cc1973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d140c700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc0ff8d0 0x7f41cc1973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d140c700 1 -- 172.21.15.62:0/3223767710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41cc1024c0 msgr2=0x7f41cc19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d140c700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41cc1024c0 0x7f41cc19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.484+0000 7f41d140c700 1 -- 172.21.15.62:0/3223767710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c4009d00 con 0x7f41cc0fef30 2024-05-08T22:35:52.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.485+0000 7f41d140c700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41cc0fef30 0x7f41cc196eb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e970 tx=0x7f41c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:52.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.485+0000 7f41be7fc700 1 -- 172.21.15.62:0/3223767710 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8009790 con 0x7f41cc0fef30 2024-05-08T22:35:52.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.485+0000 7f41be7fc700 1 -- 172.21.15.62:0/3223767710 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c800c7e0 con 0x7f41cc0fef30 2024-05-08T22:35:52.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.485+0000 7f41be7fc700 1 -- 172.21.15.62:0/3223767710 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8003c70 con 0x7f41cc0fef30 2024-05-08T22:35:52.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.485+0000 7f41d3670700 1 -- 172.21.15.62:0/3223767710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc19ef90 con 0x7f41cc0fef30 2024-05-08T22:35:52.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.485+0000 7f41d3670700 1 -- 172.21.15.62:0/3223767710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc19f3b0 con 0x7f41cc0fef30 2024-05-08T22:35:52.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.486+0000 7f41d3670700 1 -- 172.21.15.62:0/3223767710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41cc0674f0 con 0x7f41cc0fef30 2024-05-08T22:35:52.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.487+0000 7f41be7fc700 1 -- 172.21.15.62:0/3223767710 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f41c800cc50 con 0x7f41cc0fef30 2024-05-08T22:35:52.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.487+0000 7f41be7fc700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f41b80a1230 0x7f41b80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.488+0000 7f41d0c0b700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f41b80a1230 0x7f41b80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.488+0000 7f41be7fc700 1 -- 172.21.15.62:0/3223767710 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f41c8014070 con 0x7f41cc0fef30 2024-05-08T22:35:52.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.488+0000 7f41d0c0b700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f41b80a1230 0x7f41b80a36f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f41c0005950 tx=0x7f41c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:52.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.491+0000 7f41be7fc700 1 -- 172.21.15.62:0/3223767710 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f41c80961a0 con 0x7f41cc0fef30 2024-05-08T22:35:52.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.747+0000 7f41d3670700 1 -- 172.21.15.62:0/3223767710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41cc0674f0 con 0x7f41cc0fef30 2024-05-08T22:35:52.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.748+0000 7f41be7fc700 1 -- 172.21.15.62:0/3223767710 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10897 (secure 0 0 0) 0x7f41cc0674f0 con 0x7f41cc0fef30 2024-05-08T22:35:52.748 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:52.748 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:35:09.803459+0000","modified":"2024-05-08T22:35:16.261557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24529,"mds_2":24505,"mds_3":24503,"mds_4":14661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/1756743060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1756743060},{"type":"v1","addr":"172.21.15.62:6839","nonce":1756743060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14661":{"gid":14661,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/780017086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":780017086},{"type":"v1","addr":"172.21.15.62:6841","nonce":780017086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.79:6839/3534785027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":3534785027},{"type":"v1","addr":"172.21.15.79:6839","nonce":3534785027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/3474672912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":3474672912},{"type":"v1","addr":"172.21.15.66:6839","nonce":3474672912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:35:52.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 -- 172.21.15.62:0/3223767710 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f41b80a1230 msgr2=0x7f41b80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f41b80a1230 0x7f41b80a36f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f41c0005950 tx=0x7f41c000b410 comp rx=0 tx=0).stop 2024-05-08T22:35:52.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 -- 172.21.15.62:0/3223767710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41cc0fef30 msgr2=0x7f41cc196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41cc0fef30 0x7f41cc196eb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e970 tx=0x7f41c800ec80 comp rx=0 tx=0).stop 2024-05-08T22:35:52.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 -- 172.21.15.62:0/3223767710 shutdown_connections 2024-05-08T22:35:52.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f41b80a1230 0x7f41b80a36f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41cc1024c0 0x7f41cc19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41cc0fef30 0x7f41cc196eb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 --2- 172.21.15.62:0/3223767710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41cc0ff8d0 0x7f41cc1973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 -- 172.21.15.62:0/3223767710 >> 172.21.15.62:0/3223767710 conn(0x7f41cc0faa90 msgr2=0x7f41cc105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:52.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 -- 172.21.15.62:0/3223767710 shutdown_connections 2024-05-08T22:35:52.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.750+0000 7f41d3670700 1 -- 172.21.15.62:0/3223767710 wait complete. 2024-05-08T22:35:52.752 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-05-08T22:35:52.763 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:52.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.890+0000 7fd90c664700 1 -- 172.21.15.62:0/468904701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd904101910 msgr2=0x7fd9041081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.890+0000 7fd90c664700 1 --2- 172.21.15.62:0/468904701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd904101910 0x7fd9041081c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00b0d0 tx=0x7fd8fc00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.890+0000 7fd90c664700 1 -- 172.21.15.62:0/468904701 shutdown_connections 2024-05-08T22:35:52.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.890+0000 7fd90c664700 1 --2- 172.21.15.62:0/468904701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd904101910 0x7fd9041081c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.890+0000 7fd90c664700 1 --2- 172.21.15.62:0/468904701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9040ffd80 0x7fd9041001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.890+0000 7fd90c664700 1 --2- 172.21.15.62:0/468904701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd904100f70 0x7fd9041013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.890+0000 7fd90c664700 1 -- 172.21.15.62:0/468904701 >> 172.21.15.62:0/468904701 conn(0x7fd9040fb310 msgr2=0x7fd9040fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.890+0000 7fd90c664700 1 -- 172.21.15.62:0/468904701 shutdown_connections 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.890+0000 7fd90c664700 1 -- 172.21.15.62:0/468904701 wait complete. 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90c664700 1 Processor -- start 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90c664700 1 -- start start 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90c664700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9040ffd80 0x7fd904196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90c664700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd904100f70 0x7fd9041974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90c664700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd904101910 0x7fd90419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90c664700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90419ea80 con 0x7fd904101910 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90c664700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90419ebc0 con 0x7fd9040ffd80 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90c664700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90419ed50 con 0x7fd904100f70 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90ac01700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd904101910 0x7fd90419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90ac01700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd904101910 0x7fd90419e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44142/0 (socket says 172.21.15.62:44142) 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90ac01700 1 -- 172.21.15.62:0/786434660 learned_addr learned my addr 172.21.15.62:0/786434660 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd909bff700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd904100f70 0x7fd9041974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90a400700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9040ffd80 0x7fd904196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90ac01700 1 -- 172.21.15.62:0/786434660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd904100f70 msgr2=0x7fd9041974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90ac01700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd904100f70 0x7fd9041974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90ac01700 1 -- 172.21.15.62:0/786434660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9040ffd80 msgr2=0x7fd904196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90ac01700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9040ffd80 0x7fd904196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90ac01700 1 -- 172.21.15.62:0/786434660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc009d00 con 0x7fd904101910 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.891+0000 7fd90ac01700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd904101910 0x7fd90419e4b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc008ef0 tx=0x7fd8fc008f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.892+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/786434660 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc00e040 con 0x7fd904101910 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.892+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/786434660 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8fc01db30 con 0x7fd904101910 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.892+0000 7fd90c664700 1 -- 172.21.15.62:0/786434660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90419efd0 con 0x7fd904101910 2024-05-08T22:35:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.892+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/786434660 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc01ba90 con 0x7fd904101910 2024-05-08T22:35:52.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.892+0000 7fd90c664700 1 -- 172.21.15.62:0/786434660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90419f3f0 con 0x7fd904101910 2024-05-08T22:35:52.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.892+0000 7fd90c664700 1 -- 172.21.15.62:0/786434660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd904067d10 con 0x7fd904101910 2024-05-08T22:35:52.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.894+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/786434660 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd8fc01dca0 con 0x7fd904101910 2024-05-08T22:35:52.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.894+0000 7fd8fb7fe700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8f00a11b0 0x7fd8f00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:52.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.894+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/786434660 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd8fc0ce9c0 con 0x7fd904101910 2024-05-08T22:35:52.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.894+0000 7fd90a400700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8f00a11b0 0x7fd8f00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:52.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.895+0000 7fd90a400700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8f00a11b0 0x7fd8f00a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd904100dd0 tx=0x7fd90000b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:52.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:52.898+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/786434660 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd8fc098230 con 0x7fd904101910 2024-05-08T22:35:53.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.107+0000 7fd90c664700 1 -- 172.21.15.62:0/786434660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd90410b810 con 0x7fd904101910 2024-05-08T22:35:53.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.109+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/786434660 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22047 (secure 0 0 0) 0x7fd8fc02ee60 con 0x7fd904101910 2024-05-08T22:35:53.109 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:53.109 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":79,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:35:10.805199+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:53.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 -- 172.21.15.62:0/786434660 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8f00a11b0 msgr2=0x7fd8f00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8f00a11b0 0x7fd8f00a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd904100dd0 tx=0x7fd90000b5b0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 -- 172.21.15.62:0/786434660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd904101910 msgr2=0x7fd90419e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd904101910 0x7fd90419e4b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc008ef0 tx=0x7fd8fc008f20 comp rx=0 tx=0).stop 2024-05-08T22:35:53.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 -- 172.21.15.62:0/786434660 shutdown_connections 2024-05-08T22:35:53.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd904101910 0x7fd90419e4b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8f00a11b0 0x7fd8f00a3670 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9040ffd80 0x7fd904196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 --2- 172.21.15.62:0/786434660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd904100f70 0x7fd9041974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 -- 172.21.15.62:0/786434660 >> 172.21.15.62:0/786434660 conn(0x7fd9040fb310 msgr2=0x7fd9040fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:53.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 -- 172.21.15.62:0/786434660 shutdown_connections 2024-05-08T22:35:53.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.111+0000 7fd90c664700 1 -- 172.21.15.62:0/786434660 wait complete. 2024-05-08T22:35:53.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:52 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/335609501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:35:53.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:52 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/4127213897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:35:53.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:52 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3223767710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:53.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:31:54.466249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:35:08.296334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:35:09.167901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:35:53.125 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:35:53.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.254+0000 7fe2a1444700 1 -- 172.21.15.62:0/4283851781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c0ffda0 msgr2=0x7fe29c1001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.254+0000 7fe2a1444700 1 --2- 172.21.15.62:0/4283851781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c0ffda0 0x7fe29c1001c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0099c0 tx=0x7fe28c009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.254+0000 7fe2a1444700 1 -- 172.21.15.62:0/4283851781 shutdown_connections 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.254+0000 7fe2a1444700 1 --2- 172.21.15.62:0/4283851781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c101930 0x7fe29c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.254+0000 7fe2a1444700 1 --2- 172.21.15.62:0/4283851781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c0ffda0 0x7fe29c1001c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.254+0000 7fe2a1444700 1 --2- 172.21.15.62:0/4283851781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c100f90 0x7fe29c1013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.254+0000 7fe2a1444700 1 -- 172.21.15.62:0/4283851781 >> 172.21.15.62:0/4283851781 conn(0x7fe29c0fb330 msgr2=0x7fe29c0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.254+0000 7fe2a1444700 1 -- 172.21.15.62:0/4283851781 shutdown_connections 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.254+0000 7fe2a1444700 1 -- 172.21.15.62:0/4283851781 wait complete. 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe2a1444700 1 Processor -- start 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe2a1444700 1 -- start start 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe2a1444700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c0ffda0 0x7fe29c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe2a1444700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c100f90 0x7fe29c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe2a1444700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c101930 0x7fe29c19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe2a1444700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c19ea60 con 0x7fe29c100f90 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe2a1444700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c19eba0 con 0x7fe29c101930 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe2a1444700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c19ed30 con 0x7fe29c0ffda0 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe29a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c100f90 0x7fe29c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe29a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c100f90 0x7fe29c1974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44160/0 (socket says 172.21.15.62:44160) 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe29a7fc700 1 -- 172.21.15.62:0/2399169349 learned_addr learned my addr 172.21.15.62:0/2399169349 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe29affd700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c0ffda0 0x7fe29c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.255+0000 7fe29b7fe700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c101930 0x7fe29c19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.256+0000 7fe29a7fc700 1 -- 172.21.15.62:0/2399169349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c0ffda0 msgr2=0x7fe29c196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.256+0000 7fe29a7fc700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c0ffda0 0x7fe29c196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.256+0000 7fe29a7fc700 1 -- 172.21.15.62:0/2399169349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c101930 msgr2=0x7fe29c19e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.256+0000 7fe29a7fc700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c101930 0x7fe29c19e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.256+0000 7fe29a7fc700 1 -- 172.21.15.62:0/2399169349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0095f0 con 0x7fe29c100f90 2024-05-08T22:35:53.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.256+0000 7fe29a7fc700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c100f90 0x7fe29c1974c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe28400b530 tx=0x7fe28400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:53.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.256+0000 7fe293fff700 1 -- 172.21.15.62:0/2399169349 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284011830 con 0x7fe29c100f90 2024-05-08T22:35:53.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.256+0000 7fe293fff700 1 -- 172.21.15.62:0/2399169349 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe284011e50 con 0x7fe29c100f90 2024-05-08T22:35:53.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.256+0000 7fe2a1444700 1 -- 172.21.15.62:0/2399169349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c19efb0 con 0x7fe29c100f90 2024-05-08T22:35:53.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.256+0000 7fe293fff700 1 -- 172.21.15.62:0/2399169349 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28400f2e0 con 0x7fe29c100f90 2024-05-08T22:35:53.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.256+0000 7fe2a1444700 1 -- 172.21.15.62:0/2399169349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c19f510 con 0x7fe29c100f90 2024-05-08T22:35:53.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.257+0000 7fe2a1444700 1 -- 172.21.15.62:0/2399169349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c068160 con 0x7fe29c100f90 2024-05-08T22:35:53.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.258+0000 7fe293fff700 1 -- 172.21.15.62:0/2399169349 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe2840105a0 con 0x7fe29c100f90 2024-05-08T22:35:53.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.259+0000 7fe293fff700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe2880a11b0 0x7fe2880a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.259+0000 7fe293fff700 1 -- 172.21.15.62:0/2399169349 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe2840cc790 con 0x7fe29c100f90 2024-05-08T22:35:53.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.259+0000 7fe29affd700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe2880a11b0 0x7fe2880a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.259+0000 7fe29affd700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe2880a11b0 0x7fe2880a3670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009f50 tx=0x7fe28c0037c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:53.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.262+0000 7fe293fff700 1 -- 172.21.15.62:0/2399169349 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe284095f80 con 0x7fe29c100f90 2024-05-08T22:35:53.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:52 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/335609501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:35:53.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:52 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/4127213897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:35:53.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:52 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3223767710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:53.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:52 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/335609501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:35:53.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:52 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/4127213897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:35:53.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:52 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3223767710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:53.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.469+0000 7fe2a1444700 1 -- 172.21.15.62:0/2399169349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe29c10b950 con 0x7fe29c100f90 2024-05-08T22:35:53.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.470+0000 7fe293fff700 1 -- 172.21.15.62:0/2399169349 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2840995a0 con 0x7fe29c100f90 2024-05-08T22:35:53.470 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-05-08T22:35:53.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 -- 172.21.15.62:0/2399169349 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe2880a11b0 msgr2=0x7fe2880a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe2880a11b0 0x7fe2880a3670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009f50 tx=0x7fe28c0037c0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 -- 172.21.15.62:0/2399169349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c100f90 msgr2=0x7fe29c1974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c100f90 0x7fe29c1974c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe28400b530 tx=0x7fe28400b840 comp rx=0 tx=0).stop 2024-05-08T22:35:53.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 -- 172.21.15.62:0/2399169349 shutdown_connections 2024-05-08T22:35:53.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe2880a11b0 0x7fe2880a3670 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe29c101930 0x7fe29c19e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c0ffda0 0x7fe29c196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 --2- 172.21.15.62:0/2399169349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c100f90 0x7fe29c1974c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 -- 172.21.15.62:0/2399169349 >> 172.21.15.62:0/2399169349 conn(0x7fe29c0fb330 msgr2=0x7fe29c069510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:53.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 -- 172.21.15.62:0/2399169349 shutdown_connections 2024-05-08T22:35:53.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.472+0000 7fe2a1444700 1 -- 172.21.15.62:0/2399169349 wait complete. 2024-05-08T22:35:53.485 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:35:53.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.616+0000 7f9d78b03700 1 -- 172.21.15.62:0/2407272489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d74100ea0 msgr2=0x7f9d741012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.616+0000 7f9d78b03700 1 --2- 172.21.15.62:0/2407272489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d74100ea0 0x7f9d741012c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009990 tx=0x7f9d68009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.616+0000 7f9d78b03700 1 -- 172.21.15.62:0/2407272489 shutdown_connections 2024-05-08T22:35:53.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.616+0000 7f9d78b03700 1 --2- 172.21.15.62:0/2407272489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d741029b0 0x7f9d741092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.616+0000 7f9d78b03700 1 --2- 172.21.15.62:0/2407272489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d74100ea0 0x7f9d741012c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.616+0000 7f9d78b03700 1 --2- 172.21.15.62:0/2407272489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d74101ff0 0x7f9d74102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.616+0000 7f9d78b03700 1 -- 172.21.15.62:0/2407272489 >> 172.21.15.62:0/2407272489 conn(0x7f9d740fc430 msgr2=0x7f9d740fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:53.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.616+0000 7f9d78b03700 1 -- 172.21.15.62:0/2407272489 shutdown_connections 2024-05-08T22:35:53.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.616+0000 7f9d78b03700 1 -- 172.21.15.62:0/2407272489 wait complete. 2024-05-08T22:35:53.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d78b03700 1 Processor -- start 2024-05-08T22:35:53.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d78b03700 1 -- start start 2024-05-08T22:35:53.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d78b03700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d74100ea0 0x7f9d74196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d78b03700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d74101ff0 0x7f9d74197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d78b03700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d741029b0 0x7f9d7419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d78b03700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7419e940 con 0x7f9d74101ff0 2024-05-08T22:35:53.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d78b03700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7419ea80 con 0x7f9d74100ea0 2024-05-08T22:35:53.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d78b03700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7419ec10 con 0x7f9d741029b0 2024-05-08T22:35:53.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d71d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d74101ff0 0x7f9d74197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d71d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d74101ff0 0x7f9d74197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44182/0 (socket says 172.21.15.62:44182) 2024-05-08T22:35:53.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d71d9b700 1 -- 172.21.15.62:0/800683539 learned_addr learned my addr 172.21.15.62:0/800683539 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:53.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.617+0000 7f9d7259c700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d74100ea0 0x7f9d74196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d72d9d700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d741029b0 0x7f9d7419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d71d9b700 1 -- 172.21.15.62:0/800683539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d741029b0 msgr2=0x7f9d7419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d71d9b700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d741029b0 0x7f9d7419e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d71d9b700 1 -- 172.21.15.62:0/800683539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d74100ea0 msgr2=0x7f9d74196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d71d9b700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d74100ea0 0x7f9d74196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d71d9b700 1 -- 172.21.15.62:0/800683539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d680095f0 con 0x7f9d74101ff0 2024-05-08T22:35:53.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d71d9b700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d74101ff0 0x7f9d74197350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400d950 tx=0x7f9d6400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:53.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d637fe700 1 -- 172.21.15.62:0/800683539 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64009940 con 0x7f9d74101ff0 2024-05-08T22:35:53.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d637fe700 1 -- 172.21.15.62:0/800683539 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d64010460 con 0x7f9d74101ff0 2024-05-08T22:35:53.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d78b03700 1 -- 172.21.15.62:0/800683539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7419eef0 con 0x7f9d74101ff0 2024-05-08T22:35:53.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d637fe700 1 -- 172.21.15.62:0/800683539 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400f620 con 0x7f9d74101ff0 2024-05-08T22:35:53.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.618+0000 7f9d78b03700 1 -- 172.21.15.62:0/800683539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7419f370 con 0x7f9d74101ff0 2024-05-08T22:35:53.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.619+0000 7f9d78b03700 1 -- 172.21.15.62:0/800683539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d74108310 con 0x7f9d74101ff0 2024-05-08T22:35:53.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.620+0000 7f9d637fe700 1 -- 172.21.15.62:0/800683539 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9d640108d0 con 0x7f9d74101ff0 2024-05-08T22:35:53.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.621+0000 7f9d637fe700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a11b0 0x7f9d5c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.621+0000 7f9d7259c700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a11b0 0x7f9d5c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.621+0000 7f9d637fe700 1 -- 172.21.15.62:0/800683539 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9d640cc810 con 0x7f9d74101ff0 2024-05-08T22:35:53.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.621+0000 7f9d7259c700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a11b0 0x7f9d5c0a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800b5c0 tx=0x7f9d68005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:53.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.625+0000 7f9d637fe700 1 -- 172.21.15.62:0/800683539 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d64095ef0 con 0x7f9d74101ff0 2024-05-08T22:35:53.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.831+0000 7f9d78b03700 1 -- 172.21.15.62:0/800683539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d7410c880 con 0x7f9d74101ff0 2024-05-08T22:35:53.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.831+0000 7f9d637fe700 1 -- 172.21.15.62:0/800683539 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d64099510 con 0x7f9d74101ff0 2024-05-08T22:35:53.831 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-05-08T22:35:53.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 -- 172.21.15.62:0/800683539 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a11b0 msgr2=0x7f9d5c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a11b0 0x7f9d5c0a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800b5c0 tx=0x7f9d68005fd0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 -- 172.21.15.62:0/800683539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d74101ff0 msgr2=0x7f9d74197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d74101ff0 0x7f9d74197350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400d950 tx=0x7f9d6400dc60 comp rx=0 tx=0).stop 2024-05-08T22:35:53.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 -- 172.21.15.62:0/800683539 shutdown_connections 2024-05-08T22:35:53.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d741029b0 0x7f9d7419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f9d5c0a11b0 0x7f9d5c0a3670 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d74100ea0 0x7f9d74196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 --2- 172.21.15.62:0/800683539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d74101ff0 0x7f9d74197350 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 -- 172.21.15.62:0/800683539 >> 172.21.15.62:0/800683539 conn(0x7f9d740fc430 msgr2=0x7f9d74105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:53.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 -- 172.21.15.62:0/800683539 shutdown_connections 2024-05-08T22:35:53.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.834+0000 7f9d78b03700 1 -- 172.21.15.62:0/800683539 wait complete. 2024-05-08T22:35:53.846 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:35:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.976+0000 7fc39c3c8700 1 -- 172.21.15.62:0/3475673519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc394101fd0 msgr2=0x7fc394102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.976+0000 7fc39c3c8700 1 --2- 172.21.15.62:0/3475673519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc394101fd0 0x7fc394102450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc388009990 tx=0x7fc388009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.976+0000 7fc39c3c8700 1 -- 172.21.15.62:0/3475673519 shutdown_connections 2024-05-08T22:35:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.976+0000 7fc39c3c8700 1 --2- 172.21.15.62:0/3475673519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc394102990 0x7fc3941092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.976+0000 7fc39c3c8700 1 --2- 172.21.15.62:0/3475673519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc394100e80 0x7fc3941012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.976+0000 7fc39c3c8700 1 --2- 172.21.15.62:0/3475673519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc394101fd0 0x7fc394102450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.976+0000 7fc39c3c8700 1 -- 172.21.15.62:0/3475673519 >> 172.21.15.62:0/3475673519 conn(0x7fc3940fc3f0 msgr2=0x7fc3940fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:53.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.976+0000 7fc39c3c8700 1 -- 172.21.15.62:0/3475673519 shutdown_connections 2024-05-08T22:35:53.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.976+0000 7fc39c3c8700 1 -- 172.21.15.62:0/3475673519 wait complete. 2024-05-08T22:35:53.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc39c3c8700 1 Processor -- start 2024-05-08T22:35:53.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc39c3c8700 1 -- start start 2024-05-08T22:35:53.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc39c3c8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc394100e80 0x7fc394196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc39c3c8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc394101fd0 0x7fc394197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc39c3c8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc394102990 0x7fc39419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc39c3c8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39419e940 con 0x7fc394101fd0 2024-05-08T22:35:53.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc39c3c8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39419ea80 con 0x7fc394102990 2024-05-08T22:35:53.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc39c3c8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39419ec10 con 0x7fc394100e80 2024-05-08T22:35:53.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc399963700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc394101fd0 0x7fc394197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc399963700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc394101fd0 0x7fc394197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44212/0 (socket says 172.21.15.62:44212) 2024-05-08T22:35:53.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc399963700 1 -- 172.21.15.62:0/1152318846 learned_addr learned my addr 172.21.15.62:0/1152318846 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:53.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc39a965700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc394102990 0x7fc39419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.977+0000 7fc39a164700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc394100e80 0x7fc394196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.978+0000 7fc399963700 1 -- 172.21.15.62:0/1152318846 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc394100e80 msgr2=0x7fc394196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.978+0000 7fc399963700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc394100e80 0x7fc394196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.978+0000 7fc399963700 1 -- 172.21.15.62:0/1152318846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc394102990 msgr2=0x7fc39419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:53.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.978+0000 7fc399963700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc394102990 0x7fc39419e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:53.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.978+0000 7fc399963700 1 -- 172.21.15.62:0/1152318846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3880095f0 con 0x7fc394101fd0 2024-05-08T22:35:53.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.978+0000 7fc399963700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc394101fd0 0x7fc394197350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc388006010 tx=0x7fc388004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:53.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.978+0000 7fc3877fe700 1 -- 172.21.15.62:0/1152318846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38801d070 con 0x7fc394101fd0 2024-05-08T22:35:53.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.978+0000 7fc3877fe700 1 -- 172.21.15.62:0/1152318846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc388005030 con 0x7fc394101fd0 2024-05-08T22:35:53.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.978+0000 7fc39c3c8700 1 -- 172.21.15.62:0/1152318846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39419ee90 con 0x7fc394101fd0 2024-05-08T22:35:53.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.978+0000 7fc3877fe700 1 -- 172.21.15.62:0/1152318846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388004ba0 con 0x7fc394101fd0 2024-05-08T22:35:53.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.978+0000 7fc39c3c8700 1 -- 172.21.15.62:0/1152318846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39419f250 con 0x7fc394101fd0 2024-05-08T22:35:53.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.979+0000 7fc39c3c8700 1 -- 172.21.15.62:0/1152318846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3941082c0 con 0x7fc394101fd0 2024-05-08T22:35:53.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.980+0000 7fc3877fe700 1 -- 172.21.15.62:0/1152318846 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc388004d00 con 0x7fc394101fd0 2024-05-08T22:35:53.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.981+0000 7fc3877fe700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3800a11b0 0x7fc3800a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:53.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.981+0000 7fc39a164700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3800a11b0 0x7fc3800a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:53.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.981+0000 7fc3877fe700 1 -- 172.21.15.62:0/1152318846 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3880cde60 con 0x7fc394101fd0 2024-05-08T22:35:53.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.981+0000 7fc39a164700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3800a11b0 0x7fc3800a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc390009c60 tx=0x7fc390009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:53.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:53.984+0000 7fc3877fe700 1 -- 172.21.15.62:0/1152318846 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc3880975d0 con 0x7fc394101fd0 2024-05-08T22:35:54.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:53 smithi062 ceph-mon[42571]: pgmap v243: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-08T22:35:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:53 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/786434660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:53 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2399169349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:35:54.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:53 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/800683539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:35:54.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.190+0000 7fc39c3c8700 1 -- 172.21.15.62:0/1152318846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc39410c6d0 con 0x7fc394101fd0 2024-05-08T22:35:54.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.190+0000 7fc3877fe700 1 -- 172.21.15.62:0/1152318846 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc38809abf0 con 0x7fc394101fd0 2024-05-08T22:35:54.190 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-05-08T22:35:54.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 -- 172.21.15.62:0/1152318846 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3800a11b0 msgr2=0x7fc3800a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3800a11b0 0x7fc3800a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc390009c60 tx=0x7fc390009260 comp rx=0 tx=0).stop 2024-05-08T22:35:54.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 -- 172.21.15.62:0/1152318846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc394101fd0 msgr2=0x7fc394197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc394101fd0 0x7fc394197350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc388006010 tx=0x7fc388004730 comp rx=0 tx=0).stop 2024-05-08T22:35:54.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 -- 172.21.15.62:0/1152318846 shutdown_connections 2024-05-08T22:35:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc3800a11b0 0x7fc3800a3670 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc394102990 0x7fc39419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc394100e80 0x7fc394196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 --2- 172.21.15.62:0/1152318846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc394101fd0 0x7fc394197350 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 -- 172.21.15.62:0/1152318846 >> 172.21.15.62:0/1152318846 conn(0x7fc3940fc3f0 msgr2=0x7fc394105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 -- 172.21.15.62:0/1152318846 shutdown_connections 2024-05-08T22:35:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.192+0000 7fc39c3c8700 1 -- 172.21.15.62:0/1152318846 wait complete. 2024-05-08T22:35:54.205 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:54.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:53 smithi066 ceph-mon[46830]: pgmap v243: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-08T22:35:54.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:53 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/786434660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:54.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:53 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2399169349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:35:54.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:53 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/800683539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:35:54.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.331+0000 7f6767e98700 1 -- 172.21.15.62:0/2848326265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6760104290 msgr2=0x7f6760106680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.331+0000 7f6767e98700 1 --2- 172.21.15.62:0/2848326265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6760104290 0x7f6760106680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6750009990 tx=0x7f6750009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.331+0000 7f6767e98700 1 -- 172.21.15.62:0/2848326265 shutdown_connections 2024-05-08T22:35:54.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.331+0000 7f6767e98700 1 --2- 172.21.15.62:0/2848326265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760106bc0 0x7f6760108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.331+0000 7f6767e98700 1 --2- 172.21.15.62:0/2848326265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760101960 0x7f6760103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.331+0000 7f6767e98700 1 --2- 172.21.15.62:0/2848326265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6760104290 0x7f6760106680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.331+0000 7f6767e98700 1 -- 172.21.15.62:0/2848326265 >> 172.21.15.62:0/2848326265 conn(0x7f67600fb310 msgr2=0x7f67600fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:54.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.331+0000 7f6767e98700 1 -- 172.21.15.62:0/2848326265 shutdown_connections 2024-05-08T22:35:54.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.331+0000 7f6767e98700 1 -- 172.21.15.62:0/2848326265 wait complete. 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6767e98700 1 Processor -- start 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6767e98700 1 -- start start 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6767e98700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760101960 0x7f6760101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6767e98700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760104290 0x7f67600ff910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6767e98700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6760106bc0 0x7f67600ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6767e98700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67601005f0 con 0x7f6760106bc0 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6767e98700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6760100760 con 0x7f6760101960 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6767e98700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67601a0df0 con 0x7f6760104290 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6766435700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6760106bc0 0x7f67600ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6766435700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6760106bc0 0x7f67600ffe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44240/0 (socket says 172.21.15.62:44240) 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6766435700 1 -- 172.21.15.62:0/1392146354 learned_addr learned my addr 172.21.15.62:0/1392146354 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:54.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.332+0000 7f6765433700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760104290 0x7f67600ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6765c34700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760101960 0x7f6760101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6766435700 1 -- 172.21.15.62:0/1392146354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760104290 msgr2=0x7f67600ff910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6766435700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760104290 0x7f67600ff910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6766435700 1 -- 172.21.15.62:0/1392146354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760101960 msgr2=0x7f6760101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6766435700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760101960 0x7f6760101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6766435700 1 -- 172.21.15.62:0/1392146354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67500095f0 con 0x7f6760106bc0 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6765c34700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760101960 0x7f6760101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6766435700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6760106bc0 0x7f67600ffe50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f675800ef30 tx=0x7f675800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6756ffd700 1 -- 172.21.15.62:0/1392146354 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758018070 con 0x7f6760106bc0 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6756ffd700 1 -- 172.21.15.62:0/1392146354 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f675800f040 con 0x7f6760106bc0 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6756ffd700 1 -- 172.21.15.62:0/1392146354 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675800cd50 con 0x7f6760106bc0 2024-05-08T22:35:54.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6767e98700 1 -- 172.21.15.62:0/1392146354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67601a1020 con 0x7f6760106bc0 2024-05-08T22:35:54.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.333+0000 7f6767e98700 1 -- 172.21.15.62:0/1392146354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67601a14e0 con 0x7f6760106bc0 2024-05-08T22:35:54.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.334+0000 7f6767e98700 1 -- 172.21.15.62:0/1392146354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67600fce90 con 0x7f6760106bc0 2024-05-08T22:35:54.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.335+0000 7f6756ffd700 1 -- 172.21.15.62:0/1392146354 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6758003a00 con 0x7f6760106bc0 2024-05-08T22:35:54.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.336+0000 7f6756ffd700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a1230 0x7f674c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:54.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.336+0000 7f6756ffd700 1 -- 172.21.15.62:0/1392146354 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f67580ccdb0 con 0x7f6760106bc0 2024-05-08T22:35:54.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.336+0000 7f6765c34700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a1230 0x7f674c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:54.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.336+0000 7f6765c34700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a1230 0x7f674c0a36f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f675c009cb0 tx=0x7f675c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:54.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.339+0000 7f6756ffd700 1 -- 172.21.15.62:0/1392146354 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6758096620 con 0x7f6760106bc0 2024-05-08T22:35:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:53 smithi079 ceph-mon[46738]: pgmap v243: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-08T22:35:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:53 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/786434660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:53 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2399169349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:35:54.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:53 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/800683539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:35:54.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.594+0000 7f6767e98700 1 -- 172.21.15.62:0/1392146354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f676010c670 con 0x7f6760106bc0 2024-05-08T22:35:54.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.594+0000 7f6756ffd700 1 -- 172.21.15.62:0/1392146354 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10897 (secure 0 0 0) 0x7f6758099c40 con 0x7f6760106bc0 2024-05-08T22:35:54.594 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:54.595 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:35:09.803459+0000","modified":"2024-05-08T22:35:16.261557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24529,"mds_2":24505,"mds_3":24503,"mds_4":14661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/1756743060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1756743060},{"type":"v1","addr":"172.21.15.62:6839","nonce":1756743060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14661":{"gid":14661,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/780017086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":780017086},{"type":"v1","addr":"172.21.15.62:6841","nonce":780017086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.79:6839/3534785027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":3534785027},{"type":"v1","addr":"172.21.15.79:6839","nonce":3534785027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/3474672912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":3474672912},{"type":"v1","addr":"172.21.15.66:6839","nonce":3474672912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:35:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 -- 172.21.15.62:0/1392146354 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a1230 msgr2=0x7f674c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a1230 0x7f674c0a36f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f675c009cb0 tx=0x7f675c009260 comp rx=0 tx=0).stop 2024-05-08T22:35:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 -- 172.21.15.62:0/1392146354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6760106bc0 msgr2=0x7f67600ffe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6760106bc0 0x7f67600ffe50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f675800ef30 tx=0x7f675800c3a0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 -- 172.21.15.62:0/1392146354 shutdown_connections 2024-05-08T22:35:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6760106bc0 0x7f67600ffe50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f674c0a1230 0x7f674c0a36f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6760101960 0x7f6760101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 --2- 172.21.15.62:0/1392146354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6760104290 0x7f67600ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 -- 172.21.15.62:0/1392146354 >> 172.21.15.62:0/1392146354 conn(0x7f67600fb310 msgr2=0x7f67601037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 -- 172.21.15.62:0/1392146354 shutdown_connections 2024-05-08T22:35:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.597+0000 7f6767e98700 1 -- 172.21.15.62:0/1392146354 wait complete. 2024-05-08T22:35:54.598 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-05-08T22:35:54.609 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:35:54.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.735+0000 7ff733335700 1 -- 172.21.15.62:0/3692024198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c102990 msgr2=0x7ff72c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.735+0000 7ff733335700 1 --2- 172.21.15.62:0/3692024198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c102990 0x7ff72c1092b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff72400b0a0 tx=0x7ff72400b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 -- 172.21.15.62:0/3692024198 shutdown_connections 2024-05-08T22:35:54.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 --2- 172.21.15.62:0/3692024198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c102990 0x7ff72c1092b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 --2- 172.21.15.62:0/3692024198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff72c100e80 0x7ff72c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 --2- 172.21.15.62:0/3692024198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff72c101fd0 0x7ff72c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 -- 172.21.15.62:0/3692024198 >> 172.21.15.62:0/3692024198 conn(0x7ff72c0fc410 msgr2=0x7ff72c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:54.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 -- 172.21.15.62:0/3692024198 shutdown_connections 2024-05-08T22:35:54.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 -- 172.21.15.62:0/3692024198 wait complete. 2024-05-08T22:35:54.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 Processor -- start 2024-05-08T22:35:54.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 -- start start 2024-05-08T22:35:54.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c100e80 0x7ff72c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:54.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff72c101fd0 0x7ff72c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:54.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff72c102990 0x7ff72c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:54.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c074070 con 0x7ff72c100e80 2024-05-08T22:35:54.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0741e0 con 0x7ff72c101fd0 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.736+0000 7ff733335700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c1a0e50 con 0x7ff72c102990 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7310d1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c100e80 0x7ff72c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7310d1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c100e80 0x7ff72c074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44258/0 (socket says 172.21.15.62:44258) 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7310d1700 1 -- 172.21.15.62:0/2893883074 learned_addr learned my addr 172.21.15.62:0/2893883074 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7318d2700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff72c102990 0x7ff72c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7308d0700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff72c101fd0 0x7ff72c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7310d1700 1 -- 172.21.15.62:0/2893883074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff72c102990 msgr2=0x7ff72c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7310d1700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff72c102990 0x7ff72c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7310d1700 1 -- 172.21.15.62:0/2893883074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff72c101fd0 msgr2=0x7ff72c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7310d1700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff72c101fd0 0x7ff72c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7310d1700 1 -- 172.21.15.62:0/2893883074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff724009d00 con 0x7ff72c100e80 2024-05-08T22:35:54.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7310d1700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c100e80 0x7ff72c074d00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff72800c9b0 tx=0x7ff72800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7227fc700 1 -- 172.21.15.62:0/2893883074 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728007aa0 con 0x7ff72c100e80 2024-05-08T22:35:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff733335700 1 -- 172.21.15.62:0/2893883074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c1a10e0 con 0x7ff72c100e80 2024-05-08T22:35:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7227fc700 1 -- 172.21.15.62:0/2893883074 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff728007c00 con 0x7ff72c100e80 2024-05-08T22:35:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff7227fc700 1 -- 172.21.15.62:0/2893883074 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728018660 con 0x7ff72c100e80 2024-05-08T22:35:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.737+0000 7ff733335700 1 -- 172.21.15.62:0/2893883074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c1a15f0 con 0x7ff72c100e80 2024-05-08T22:35:54.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.738+0000 7ff733335700 1 -- 172.21.15.62:0/2893883074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c108220 con 0x7ff72c100e80 2024-05-08T22:35:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.740+0000 7ff7227fc700 1 -- 172.21.15.62:0/2893883074 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff72800f450 con 0x7ff72c100e80 2024-05-08T22:35:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.740+0000 7ff7227fc700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff7180a11b0 0x7ff7180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.741+0000 7ff7308d0700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff7180a11b0 0x7ff7180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.741+0000 7ff7227fc700 1 -- 172.21.15.62:0/2893883074 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff7280ccbb0 con 0x7ff72c100e80 2024-05-08T22:35:54.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.741+0000 7ff7308d0700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff7180a11b0 0x7ff7180a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff71c005950 tx=0x7ff71c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:54.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.744+0000 7ff7227fc700 1 -- 172.21.15.62:0/2893883074 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff728096320 con 0x7ff72c100e80 2024-05-08T22:35:54.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.954+0000 7ff733335700 1 -- 172.21.15.62:0/2893883074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff72c10c680 con 0x7ff72c100e80 2024-05-08T22:35:54.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.955+0000 7ff7227fc700 1 -- 172.21.15.62:0/2893883074 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22047 (secure 0 0 0) 0x7ff728099940 con 0x7ff72c100e80 2024-05-08T22:35:54.955 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:54.956 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":79,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:35:10.805199+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:35:54.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.957+0000 7ff733335700 1 -- 172.21.15.62:0/2893883074 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff7180a11b0 msgr2=0x7ff7180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.957+0000 7ff733335700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff7180a11b0 0x7ff7180a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff71c005950 tx=0x7ff71c0058e0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.957+0000 7ff733335700 1 -- 172.21.15.62:0/2893883074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c100e80 msgr2=0x7ff72c074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:54.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.957+0000 7ff733335700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c100e80 0x7ff72c074d00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff72800c9b0 tx=0x7ff72800ccc0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.957+0000 7ff733335700 1 -- 172.21.15.62:0/2893883074 shutdown_connections 2024-05-08T22:35:54.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.957+0000 7ff733335700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff72c102990 0x7ff72c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.957+0000 7ff733335700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c100e80 0x7ff72c074d00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.957+0000 7ff733335700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff7180a11b0 0x7ff7180a3670 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.957+0000 7ff733335700 1 --2- 172.21.15.62:0/2893883074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff72c101fd0 0x7ff72c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:54.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.957+0000 7ff733335700 1 -- 172.21.15.62:0/2893883074 >> 172.21.15.62:0/2893883074 conn(0x7ff72c0fc410 msgr2=0x7ff72c105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:54.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.958+0000 7ff733335700 1 -- 172.21.15.62:0/2893883074 shutdown_connections 2024-05-08T22:35:54.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:54.958+0000 7ff733335700 1 -- 172.21.15.62:0/2893883074 wait complete. 2024-05-08T22:35:54.971 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:35:55.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.098+0000 7faf28147700 1 -- 172.21.15.62:0/2214583842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf20100f70 msgr2=0x7faf201013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.098+0000 7faf28147700 1 --2- 172.21.15.62:0/2214583842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf20100f70 0x7faf201013d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0099c0 tx=0x7faf1c009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.098+0000 7faf28147700 1 -- 172.21.15.62:0/2214583842 shutdown_connections 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.098+0000 7faf28147700 1 --2- 172.21.15.62:0/2214583842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf20101910 0x7faf201081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.098+0000 7faf28147700 1 --2- 172.21.15.62:0/2214583842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf200ffd80 0x7faf201001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.098+0000 7faf28147700 1 --2- 172.21.15.62:0/2214583842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf20100f70 0x7faf201013d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.098+0000 7faf28147700 1 -- 172.21.15.62:0/2214583842 >> 172.21.15.62:0/2214583842 conn(0x7faf200fb2f0 msgr2=0x7faf200fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.098+0000 7faf28147700 1 -- 172.21.15.62:0/2214583842 shutdown_connections 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.098+0000 7faf28147700 1 -- 172.21.15.62:0/2214583842 wait complete. 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.098+0000 7faf28147700 1 Processor -- start 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.098+0000 7faf28147700 1 -- start start 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf28147700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf200ffd80 0x7faf20069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf28147700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf20100f70 0x7faf2019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf28147700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf20101910 0x7faf2019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf28147700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2019e8f0 con 0x7faf20101910 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf28147700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2019ea60 con 0x7faf200ffd80 2024-05-08T22:35:55.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf28147700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2019ec20 con 0x7faf20100f70 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf266e4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf20101910 0x7faf2019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf266e4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf20101910 0x7faf2019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44280/0 (socket says 172.21.15.62:44280) 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf266e4700 1 -- 172.21.15.62:0/4141835883 learned_addr learned my addr 172.21.15.62:0/4141835883 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf25ee3700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf200ffd80 0x7faf20069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf256e2700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf20100f70 0x7faf2019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf266e4700 1 -- 172.21.15.62:0/4141835883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf20100f70 msgr2=0x7faf2019bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf266e4700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf20100f70 0x7faf2019bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf266e4700 1 -- 172.21.15.62:0/4141835883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf200ffd80 msgr2=0x7faf20069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf266e4700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf200ffd80 0x7faf20069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf266e4700 1 -- 172.21.15.62:0/4141835883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c0095f0 con 0x7faf20101910 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf256e2700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf20100f70 0x7faf2019bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf266e4700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf20101910 0x7faf2019e150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faf1800b970 tx=0x7faf1800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf12ffd700 1 -- 172.21.15.62:0/4141835883 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800c930 con 0x7faf20101910 2024-05-08T22:35:55.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf12ffd700 1 -- 172.21.15.62:0/4141835883 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf18014440 con 0x7faf20101910 2024-05-08T22:35:55.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf28147700 1 -- 172.21.15.62:0/4141835883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2019ef00 con 0x7faf20101910 2024-05-08T22:35:55.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.099+0000 7faf12ffd700 1 -- 172.21.15.62:0/4141835883 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf180075d0 con 0x7faf20101910 2024-05-08T22:35:55.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.100+0000 7faf28147700 1 -- 172.21.15.62:0/4141835883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2019f380 con 0x7faf20101910 2024-05-08T22:35:55.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.100+0000 7faf28147700 1 -- 172.21.15.62:0/4141835883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf201071b0 con 0x7faf20101910 2024-05-08T22:35:55.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.102+0000 7faf12ffd700 1 -- 172.21.15.62:0/4141835883 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faf180145b0 con 0x7faf20101910 2024-05-08T22:35:55.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.102+0000 7faf12ffd700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faf0c0a1230 0x7faf0c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.102+0000 7faf25ee3700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faf0c0a1230 0x7faf0c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.102+0000 7faf12ffd700 1 -- 172.21.15.62:0/4141835883 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faf18006e40 con 0x7faf20101910 2024-05-08T22:35:55.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.103+0000 7faf25ee3700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faf0c0a1230 0x7faf0c0a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faf20100dd0 tx=0x7faf14007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.106+0000 7faf12ffd700 1 -- 172.21.15.62:0/4141835883 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faf18095b50 con 0x7faf20101910 2024-05-08T22:35:55.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:54 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1152318846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:35:55.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:54 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1392146354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.365+0000 7faf28147700 1 -- 172.21.15.62:0/4141835883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf20067480 con 0x7faf20101910 2024-05-08T22:35:55.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.366+0000 7faf12ffd700 1 -- 172.21.15.62:0/4141835883 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10897 (secure 0 0 0) 0x7faf18023090 con 0x7faf20101910 2024-05-08T22:35:55.366 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:35:55.366 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:35:09.803459+0000","modified":"2024-05-08T22:35:16.261557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24529,"mds_2":24505,"mds_3":24503,"mds_4":14661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/1756743060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1756743060},{"type":"v1","addr":"172.21.15.62:6839","nonce":1756743060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14661":{"gid":14661,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/780017086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":780017086},{"type":"v1","addr":"172.21.15.62:6841","nonce":780017086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.79:6839/3534785027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":3534785027},{"type":"v1","addr":"172.21.15.79:6839","nonce":3534785027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/3474672912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":3474672912},{"type":"v1","addr":"172.21.15.66:6839","nonce":3474672912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:35:55.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 -- 172.21.15.62:0/4141835883 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faf0c0a1230 msgr2=0x7faf0c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faf0c0a1230 0x7faf0c0a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faf20100dd0 tx=0x7faf14007400 comp rx=0 tx=0).stop 2024-05-08T22:35:55.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 -- 172.21.15.62:0/4141835883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf20101910 msgr2=0x7faf2019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf20101910 0x7faf2019e150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faf1800b970 tx=0x7faf1800bc80 comp rx=0 tx=0).stop 2024-05-08T22:35:55.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 -- 172.21.15.62:0/4141835883 shutdown_connections 2024-05-08T22:35:55.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf20101910 0x7faf2019e150 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faf0c0a1230 0x7faf0c0a36f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf200ffd80 0x7faf20069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 --2- 172.21.15.62:0/4141835883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf20100f70 0x7faf2019bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 -- 172.21.15.62:0/4141835883 >> 172.21.15.62:0/4141835883 conn(0x7faf200fb2f0 msgr2=0x7faf201042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 -- 172.21.15.62:0/4141835883 shutdown_connections 2024-05-08T22:35:55.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.368+0000 7faf28147700 1 -- 172.21.15.62:0/4141835883 wait complete. 2024-05-08T22:35:55.370 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-05-08T22:35:55.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:54 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1152318846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:35:55.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:54 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1392146354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:55.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T22:35:09.803459+0000', 'modified': '2024-05-08T22:35:16.261557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24533, 'mds_1': 24529, 'mds_2': 24505, 'mds_3': 24503, 'mds_4': 14661}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14649': {'gid': 14649, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6839/1756743060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 1756743060}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 1756743060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14661': {'gid': 14661, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.62:6841/780017086', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6840', 'nonce': 780017086}, {'type': 'v1', 'addr': '172.21.15.62:6841', 'nonce': 780017086}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24455': {'gid': 24455, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6833/4065089721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 4065089721}, {'type': 'v1', 'addr': '172.21.15.66:6833', 'nonce': 4065089721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24466': {'gid': 24466, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6833/1379725080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 1379725080}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 1379725080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6835/2449812673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 2449812673}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 2449812673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6835/2938225588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 2938225588}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 2938225588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24503': {'gid': 24503, 'name': 'h', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.66:6837/1178845415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 1178845415}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 1178845415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.79:6837/753714978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 753714978}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 753714978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.79:6839/3534785027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 3534785027}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 3534785027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24533': {'gid': 24533, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6839/3474672912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 3474672912}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 3474672912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:35:55.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-08T22:35:55.381 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T22:35:55.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:54 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1152318846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:35:55.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:54 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1392146354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:55.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.508+0000 7fae00b56700 1 -- 172.21.15.62:0/157356223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadfc101910 msgr2=0x7fadfc103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.508+0000 7fae00b56700 1 --2- 172.21.15.62:0/157356223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadfc101910 0x7fadfc103d00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fadec009990 tx=0x7fadec009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 -- 172.21.15.62:0/157356223 shutdown_connections 2024-05-08T22:35:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 --2- 172.21.15.62:0/157356223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadfc106b70 0x7fadfc108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 --2- 172.21.15.62:0/157356223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadfc101910 0x7fadfc103d00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 --2- 172.21.15.62:0/157356223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadfc104240 0x7fadfc106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 -- 172.21.15.62:0/157356223 >> 172.21.15.62:0/157356223 conn(0x7fadfc0fb300 msgr2=0x7fadfc0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 -- 172.21.15.62:0/157356223 shutdown_connections 2024-05-08T22:35:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 -- 172.21.15.62:0/157356223 wait complete. 2024-05-08T22:35:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 Processor -- start 2024-05-08T22:35:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 -- start start 2024-05-08T22:35:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadfc101910 0x7fadfc101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadfc104240 0x7fadfc0ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadfc106b70 0x7fadfc0ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1005a0 con 0x7fadfc106b70 2024-05-08T22:35:55.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.509+0000 7fae00b56700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc100710 con 0x7fadfc104240 2024-05-08T22:35:55.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fae00b56700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1a0e70 con 0x7fadfc101910 2024-05-08T22:35:55.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadfad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadfc106b70 0x7fadfc0ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadfad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadfc106b70 0x7fadfc0ffe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44290/0 (socket says 172.21.15.62:44290) 2024-05-08T22:35:55.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadfa59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadfc101910 0x7fadfc101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadfad9d700 1 -- 172.21.15.62:0/2197423986 learned_addr learned my addr 172.21.15.62:0/2197423986 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadf9d9b700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadfc104240 0x7fadfc0ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadfad9d700 1 -- 172.21.15.62:0/2197423986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadfc101910 msgr2=0x7fadfc101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadfad9d700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadfc101910 0x7fadfc101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadfad9d700 1 -- 172.21.15.62:0/2197423986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadfc104240 msgr2=0x7fadfc0ff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadfad9d700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadfc104240 0x7fadfc0ff8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadfad9d700 1 -- 172.21.15.62:0/2197423986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec0095f0 con 0x7fadfc106b70 2024-05-08T22:35:55.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadfad9d700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadfc106b70 0x7fadfc0ffe00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b970 tx=0x7fadf400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadeb7fe700 1 -- 172.21.15.62:0/2197423986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf400f860 con 0x7fadfc106b70 2024-05-08T22:35:55.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadeb7fe700 1 -- 172.21.15.62:0/2197423986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadf400fe80 con 0x7fadfc106b70 2024-05-08T22:35:55.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fae00b56700 1 -- 172.21.15.62:0/2197423986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc1a1100 con 0x7fadfc106b70 2024-05-08T22:35:55.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fadeb7fe700 1 -- 172.21.15.62:0/2197423986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf400d3f0 con 0x7fadfc106b70 2024-05-08T22:35:55.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.510+0000 7fae00b56700 1 -- 172.21.15.62:0/2197423986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc1a15c0 con 0x7fadfc106b70 2024-05-08T22:35:55.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.513+0000 7fadeb7fe700 1 -- 172.21.15.62:0/2197423986 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fadf4017600 con 0x7fadfc106b70 2024-05-08T22:35:55.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.513+0000 7fadeb7fe700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fade40a11b0 0x7fade40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.513+0000 7fadfa59c700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fade40a11b0 0x7fade40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.513+0000 7fadeb7fe700 1 -- 172.21.15.62:0/2197423986 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fadf4006e40 con 0x7fadfc106b70 2024-05-08T22:35:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.514+0000 7fadfa59c700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fade40a11b0 0x7fade40a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fadec00b5c0 tx=0x7fadec003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.514+0000 7fae0cd10b80 1 Processor -- start 2024-05-08T22:35:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.514+0000 7fae0cd10b80 1 -- start start 2024-05-08T22:35:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.514+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8b4f0 0x55e71cd8b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.514+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8be50 0x55e71cd8c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.514+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8c810 0x55e71cd8cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.514+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71cce2100 con 0x55e71cd8c810 2024-05-08T22:35:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.514+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71ccdfac0 con 0x55e71cd8be50 2024-05-08T22:35:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.514+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71ccdb3b0 con 0x55e71cd8b4f0 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadfad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8c810 0x55e71cd8cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadfad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8c810 0x55e71cd8cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44300/0 (socket says 172.21.15.62:44300) 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadf9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8be50 0x55e71cd8c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadfad9d700 1 -- 172.21.15.62:0/514695704 learned_addr learned my addr 172.21.15.62:0/514695704 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadfa59c700 1 --2- 172.21.15.62:0/514695704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8b4f0 0x55e71cd8b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadf9d9b700 1 -- 172.21.15.62:0/514695704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8b4f0 msgr2=0x55e71cd8b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadf9d9b700 1 --2- 172.21.15.62:0/514695704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8b4f0 0x55e71cd8b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadf9d9b700 1 -- 172.21.15.62:0/514695704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8c810 msgr2=0x55e71cd8cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadf9d9b700 1 --2- 172.21.15.62:0/514695704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8c810 0x55e71cd8cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadf9d9b700 1 -- 172.21.15.62:0/514695704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e71cb5c360 con 0x55e71cd8be50 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadfad9d700 1 --2- 172.21.15.62:0/514695704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8c810 0x55e71cd8cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fadf9d9b700 1 --2- 172.21.15.62:0/514695704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8be50 0x55e71cd8c2d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fadf000bcb0 tx=0x7fadf00103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90150c3d6b5b619d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fade8ff9700 1 -- 172.21.15.62:0/514695704 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0010d10 con 0x55e71cd8be50 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fade8ff9700 1 -- 172.21.15.62:0/514695704 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadf0015070 con 0x55e71cd8be50 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.515+0000 7fade8ff9700 1 -- 172.21.15.62:0/514695704 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0005020 con 0x55e71cd8be50 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.516+0000 7fae0cd10b80 1 -- 172.21.15.62:0/514695704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8be50 msgr2=0x55e71cd8c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.516+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/514695704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8be50 0x55e71cd8c2d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fadf000bcb0 tx=0x7fadf00103d0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.516+0000 7fae0cd10b80 1 -- 172.21.15.62:0/514695704 shutdown_connections 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.516+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/514695704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8c810 0x55e71cd8cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.516+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/514695704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8b4f0 0x55e71cd8b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.516+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/514695704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8be50 0x55e71cd8c2d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.516+0000 7fae0cd10b80 1 -- 172.21.15.62:0/514695704 >> 172.21.15.62:0/514695704 conn(0x55e71cce91c0 msgr2=0x55e71cd85d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.516+0000 7fae0cd10b80 1 -- 172.21.15.62:0/514695704 shutdown_connections 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.516+0000 7fae0cd10b80 1 -- 172.21.15.62:0/514695704 wait complete. 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.516+0000 7fae0cd10b80 1 -- start start 2024-05-08T22:35:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fae0cd10b80 1 Processor -- start 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 0x55e71cd94dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd95310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71cc77510 con 0x55e71cd8be50 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71cc771b0 con 0x55e71cd8b4f0 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71cb5e650 con 0x55e71cd8c810 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fae0cd10b80 10 client.0 _ll_register_callbacks cb 0x55e71cc80ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadf9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd95310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadf9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd95310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44314/0 (socket says 172.21.15.62:44314) 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadf9d9b700 1 -- 172.21.15.62:0/3120017915 learned_addr learned my addr 172.21.15.62:0/3120017915 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadfad9d700 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadfa59c700 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 0x55e71cd94dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadf9d9b700 1 -- 172.21.15.62:0/3120017915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 msgr2=0x55e71cd9c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadf9d9b700 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadf9d9b700 1 -- 172.21.15.62:0/3120017915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 msgr2=0x55e71cd94dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadf9d9b700 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 0x55e71cd94dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadf9d9b700 1 -- 172.21.15.62:0/3120017915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc1a15c0 con 0x55e71cd8be50 2024-05-08T22:35:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadfa59c700 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 0x55e71cd94dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fadf9d9b700 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd95310 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fadf0010d40 tx=0x7fadf0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f33eb389684b3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.517+0000 7fade1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:35:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fade1ffb700 1 -- 172.21.15.62:0/3120017915 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0020070 con 0x55e71cd8be50 2024-05-08T22:35:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fade1ffb700 1 -- 172.21.15.62:0/3120017915 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fadf0017070 con 0x55e71cd8be50 2024-05-08T22:35:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fae0cd10b80 1 -- 172.21.15.62:0/3120017915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e71cc34220 con 0x55e71cd8be50 2024-05-08T22:35:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fade1ffb700 1 -- 172.21.15.62:0/3120017915 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadf0004250 con 0x55e71cd8be50 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fae0cd10b80 10 client.15081 fetch_fsmap learned FSMap version 26 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fae0cd10b80 1 -- 172.21.15.62:0/3120017915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fadf000b8e0 con 0x55e71cd8be50 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fade1ffb700 1 -- 172.21.15.62:0/3120017915 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e71cc34220 con 0x55e71cd8be50 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fade1ffb700 1 -- 172.21.15.62:0/3120017915 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fadf002f070 con 0x55e71cd8be50 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fae0cd10b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fae0cd10b80 10 client.15081 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fae0cd10b80 20 client.15081 populate_metadata read hostname 'smithi062' 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x55e71cd9fed0 0x55e71cda51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fae0cd10b80 4 client.15081 mds_command: new command op to 14661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fae0cd10b80 1 -- 172.21.15.62:0/3120017915 --> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e71cd5a7c0 con 0x55e71cd9fed0 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.518+0000 7fadfa59c700 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x55e71cd9fed0 0x55e71cda51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.519+0000 7fadfa59c700 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x55e71cd9fed0 0x55e71cda51f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=863041597b678f8 server_cookie=701b9db689b54ca in_seq=0 out_seq=0 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.519+0000 7fade1ffb700 10 client.15081 ms_handle_connect on v2:172.21.15.62:6840/780017086 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.519+0000 7fade1ffb700 1 -- 172.21.15.62:0/3120017915 <== mds.4 v2:172.21.15.62:6840/780017086 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e71cd5a7c0 con 0x55e71cd9fed0 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.519+0000 7fade1ffb700 10 client.15081 handle_command_reply: tid=0 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.519+0000 7fae0cd10b80 1 client.15081 shutdown 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fae0cd10b80 1 -- 172.21.15.62:0/3120017915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 msgr2=0x55e71cd95310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd95310 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fadf0010d40 tx=0x7fadf0010d70 comp rx=0 tx=0).stop 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fae0cd10b80 1 -- 172.21.15.62:0/3120017915 shutdown_connections 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x55e71cd9fed0 0x55e71cda51f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 0x55e71cd94dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/3120017915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd95310 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fade1ffb700 0 client.15081 ms_handle_reset on v2:172.21.15.62:6840/780017086 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fae0cd10b80 1 -- 172.21.15.62:0/3120017915 >> 172.21.15.62:0/3120017915 conn(0x55e71cce91c0 msgr2=0x55e71cd85550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fae0cd10b80 1 -- 172.21.15.62:0/3120017915 shutdown_connections 2024-05-08T22:35:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fae0cd10b80 1 -- 172.21.15.62:0/3120017915 wait complete. 2024-05-08T22:35:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.520+0000 7fae0cd10b80 20 client.15081 trim_cache size 0 max 16384 2024-05-08T22:35:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fae0cd10b80 1 Processor -- start 2024-05-08T22:35:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fae0cd10b80 1 -- start start 2024-05-08T22:35:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8b4f0 0x55e71cd8b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8be50 0x55e71cd94350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71cd94940 con 0x55e71cd8b4f0 2024-05-08T22:35:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71cd82d70 con 0x55e71cd8be50 2024-05-08T22:35:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71cd9c7b0 con 0x55e71cd8c810 2024-05-08T22:35:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fadfa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8b4f0 0x55e71cd8b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fadfa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8b4f0 0x55e71cd8b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44316/0 (socket says 172.21.15.62:44316) 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fadfa59c700 1 -- 172.21.15.62:0/1253638600 learned_addr learned my addr 172.21.15.62:0/1253638600 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fadf9d9b700 1 --2- 172.21.15.62:0/1253638600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8be50 0x55e71cd94350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fadfad9d700 1 --2- 172.21.15.62:0/1253638600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fadfa59c700 1 -- 172.21.15.62:0/1253638600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 msgr2=0x55e71cd9c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fadfa59c700 1 --2- 172.21.15.62:0/1253638600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fadfa59c700 1 -- 172.21.15.62:0/1253638600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8be50 msgr2=0x55e71cd94350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fadfa59c700 1 --2- 172.21.15.62:0/1253638600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8be50 0x55e71cd94350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fadfa59c700 1 -- 172.21.15.62:0/1253638600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e71cb08b10 con 0x55e71cd8b4f0 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.538+0000 7fadfad9d700 1 --2- 172.21.15.62:0/1253638600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fadfa59c700 1 --2- 172.21.15.62:0/1253638600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8b4f0 0x55e71cd8b8d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fadec00bb70 tx=0x7fadec020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1300907ac852bca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fade97fa700 1 -- 172.21.15.62:0/1253638600 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec004910 con 0x55e71cd8b4f0 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fade97fa700 1 -- 172.21.15.62:0/1253638600 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadec02c040 con 0x55e71cd8b4f0 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fade97fa700 1 -- 172.21.15.62:0/1253638600 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec00f660 con 0x55e71cd8b4f0 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1253638600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8b4f0 msgr2=0x55e71cd8b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/1253638600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8b4f0 0x55e71cd8b8d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fadec00bb70 tx=0x7fadec020000 comp rx=0 tx=0).stop 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1253638600 shutdown_connections 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/1253638600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/1253638600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8b4f0 0x55e71cd8b8d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/1253638600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8be50 0x55e71cd94350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1253638600 >> 172.21.15.62:0/1253638600 conn(0x55e71cce91c0 msgr2=0x55e71cd8cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1253638600 shutdown_connections 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.539+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1253638600 wait complete. 2024-05-08T22:35:55.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fae0cd10b80 1 -- start start 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fae0cd10b80 1 Processor -- start 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 0x55e71cd992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd9b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fae0cd10b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71cd9c300 con 0x55e71cd8be50 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71cd9c440 con 0x55e71cd8b4f0 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fae0cd10b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71cd9c600 con 0x55e71cd8c810 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fae0cd10b80 10 client.0 _ll_register_callbacks cb 0x55e71cc80ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fadf9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd9b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fadf9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd9b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44322/0 (socket says 172.21.15.62:44322) 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fadf9d9b700 1 -- 172.21.15.62:0/1531321554 learned_addr learned my addr 172.21.15.62:0/1531321554 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fadfa59c700 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 0x55e71cd992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fadfad9d700 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fadf9d9b700 1 -- 172.21.15.62:0/1531321554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 msgr2=0x55e71cd9bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fadf9d9b700 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fadf9d9b700 1 -- 172.21.15.62:0/1531321554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 msgr2=0x55e71cd992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fadf9d9b700 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 0x55e71cd992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fadf9d9b700 1 -- 172.21.15.62:0/1531321554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc109bd0 con 0x55e71cd8be50 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.540+0000 7fadfa59c700 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 0x55e71cd992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fadf9d9b700 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd9b7f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55e71cd86950 tx=0x7fadf000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f00342eb0eeec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fade1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fade1ffb700 1 -- 172.21.15.62:0/1531321554 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0020040 con 0x55e71cd8be50 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fade1ffb700 1 -- 172.21.15.62:0/1531321554 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fadf0017040 con 0x55e71cd8be50 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1531321554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadf0015070 con 0x55e71cd8be50 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fade1ffb700 1 -- 172.21.15.62:0/1531321554 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadf0010e90 con 0x55e71cd8be50 2024-05-08T22:35:55.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fae0cd10b80 10 client.15087 fetch_fsmap learned FSMap version 26 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1531321554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e71cca1510 con 0x55e71cd8be50 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fade1ffb700 1 -- 172.21.15.62:0/1531321554 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadf0015070 con 0x55e71cd8be50 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fade1ffb700 1 -- 172.21.15.62:0/1531321554 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fadf0010440 con 0x55e71cd8be50 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fae0cd10b80 10 client.15087 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fae0cd10b80 10 client.15087 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fae0cd10b80 20 client.15087 populate_metadata read hostname 'smithi062' 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x55e71cdfb710 0x55e71cdf80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fae0cd10b80 4 client.15087 mds_command: new command op to 14661 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1531321554 --> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e71ccd3e70 con 0x55e71cdfb710 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.541+0000 7fadfa59c700 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x55e71cdfb710 0x55e71cdf80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fade1ffb700 10 client.15087 ms_handle_connect on v2:172.21.15.62:6840/780017086 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fadfa59c700 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x55e71cdfb710 0x55e71cdf80a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=68d8b5463368ec06 server_cookie=bae5b4c991ec2ba5 in_seq=0 out_seq=0 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fade1ffb700 1 -- 172.21.15.62:0/1531321554 <== mds.4 v2:172.21.15.62:6840/780017086 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e71ccd3e70 con 0x55e71cdfb710 2024-05-08T22:35:55.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fade1ffb700 10 client.15087 handle_command_reply: tid=0 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 1 client.15087 shutdown 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1531321554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 msgr2=0x55e71cd9b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd9b7f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55e71cd86950 tx=0x7fadf000b710 comp rx=0 tx=0).stop 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1531321554 shutdown_connections 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e71cd8c810 0x55e71cd9bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x55e71cdfb710 0x55e71cdf80a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e71cd8b4f0 0x55e71cd992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fade1ffb700 0 client.15087 ms_handle_reset on v2:172.21.15.62:6840/780017086 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 1 --2- 172.21.15.62:0/1531321554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71cd8be50 0x55e71cd9b7f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1531321554 >> 172.21.15.62:0/1531321554 conn(0x55e71cce91c0 msgr2=0x55e71cd92d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1531321554 shutdown_connections 2024-05-08T22:35:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 1 -- 172.21.15.62:0/1531321554 wait complete. 2024-05-08T22:35:55.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.542+0000 7fae0cd10b80 20 client.15087 trim_cache size 0 max 16384 2024-05-08T22:35:55.551 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:35:55.551 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:35:55.551 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 4, 2024-05-08T22:35:55.551 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 14661, 2024-05-08T22:35:55.551 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-05-08T22:35:55.551 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-05-08T22:35:55.551 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-05-08T22:35:55.551 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 40.286608860000001, 2024-05-08T22:35:55.552 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 26, 2024-05-08T22:35:55.552 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 79, 2024-05-08T22:35:55.552 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T22:35:55.552 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 53.279563805999999 2024-05-08T22:35:55.552 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:35:55.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 -- 172.21.15.62:0/2197423986 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fade40a11b0 msgr2=0x7fade40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fade40a11b0 0x7fade40a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fadec00b5c0 tx=0x7fadec003790 comp rx=0 tx=0).stop 2024-05-08T22:35:55.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 -- 172.21.15.62:0/2197423986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadfc106b70 msgr2=0x7fadfc0ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadfc106b70 0x7fadfc0ffe00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b970 tx=0x7fadf400bc80 comp rx=0 tx=0).stop 2024-05-08T22:35:55.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 -- 172.21.15.62:0/2197423986 shutdown_connections 2024-05-08T22:35:55.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadfc106b70 0x7fadfc0ffe00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadfc101910 0x7fadfc101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fade40a11b0 0x7fade40a3670 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 --2- 172.21.15.62:0/2197423986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fadfc104240 0x7fadfc0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 -- 172.21.15.62:0/2197423986 >> 172.21.15.62:0/2197423986 conn(0x7fadfc0fb300 msgr2=0x7fadfc102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 -- 172.21.15.62:0/2197423986 shutdown_connections 2024-05-08T22:35:55.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.543+0000 7fae00b56700 1 -- 172.21.15.62:0/2197423986 wait complete. 2024-05-08T22:35:55.555 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-08T22:35:55.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 -- 172.21.15.62:0/704845381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac100f60 msgr2=0x7f68ac1013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 --2- 172.21.15.62:0/704845381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac100f60 0x7f68ac1013c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f68a80099c0 tx=0x7f68a8009cd0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 -- 172.21.15.62:0/704845381 shutdown_connections 2024-05-08T22:35:55.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 --2- 172.21.15.62:0/704845381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac101900 0x7f68ac1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 --2- 172.21.15.62:0/704845381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68ac0ffd70 0x7f68ac100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 --2- 172.21.15.62:0/704845381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac100f60 0x7f68ac1013c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 -- 172.21.15.62:0/704845381 >> 172.21.15.62:0/704845381 conn(0x7f68ac0fb2a0 msgr2=0x7f68ac0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 -- 172.21.15.62:0/704845381 shutdown_connections 2024-05-08T22:35:55.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 -- 172.21.15.62:0/704845381 wait complete. 2024-05-08T22:35:55.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 Processor -- start 2024-05-08T22:35:55.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 -- start start 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac0ffd70 0x7f68ac196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.683+0000 7f68b4576700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68ac100f60 0x7f68ac1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b4576700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac101900 0x7f68ac19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b4576700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19e850 con 0x7f68ac101900 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b4576700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19e9c0 con 0x7f68ac100f60 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b4576700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19eb80 con 0x7f68ac0ffd70 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b2b13700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac101900 0x7f68ac19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b2b13700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac101900 0x7f68ac19e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44342/0 (socket says 172.21.15.62:44342) 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b2b13700 1 -- 172.21.15.62:0/1496743085 learned_addr learned my addr 172.21.15.62:0/1496743085 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b1b11700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68ac100f60 0x7f68ac1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b2312700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac0ffd70 0x7f68ac196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b2b13700 1 -- 172.21.15.62:0/1496743085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac0ffd70 msgr2=0x7f68ac196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b2b13700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac0ffd70 0x7f68ac196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b2b13700 1 -- 172.21.15.62:0/1496743085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68ac100f60 msgr2=0x7f68ac1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b2b13700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68ac100f60 0x7f68ac1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b2b13700 1 -- 172.21.15.62:0/1496743085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a80095f0 con 0x7f68ac101900 2024-05-08T22:35:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b2b13700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac101900 0x7f68ac19e280 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f68a4007ee0 tx=0x7f68a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68a37fe700 1 -- 172.21.15.62:0/1496743085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4017070 con 0x7f68ac101900 2024-05-08T22:35:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b4576700 1 -- 172.21.15.62:0/1496743085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac19ee60 con 0x7f68ac101900 2024-05-08T22:35:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68a37fe700 1 -- 172.21.15.62:0/1496743085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68a400f040 con 0x7f68ac101900 2024-05-08T22:35:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68a37fe700 1 -- 172.21.15.62:0/1496743085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4005580 con 0x7f68ac101900 2024-05-08T22:35:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.684+0000 7f68b4576700 1 -- 172.21.15.62:0/1496743085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac19f280 con 0x7f68ac101900 2024-05-08T22:35:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.686+0000 7f68a37fe700 1 -- 172.21.15.62:0/1496743085 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f68a4005760 con 0x7f68ac101900 2024-05-08T22:35:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.687+0000 7f68a37fe700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f68980a11b0 0x7f68980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.687+0000 7f68b2312700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f68980a11b0 0x7f68980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.687+0000 7f68a37fe700 1 -- 172.21.15.62:0/1496743085 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f68a40cc170 con 0x7f68ac101900 2024-05-08T22:35:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.688+0000 7f68b2312700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f68980a11b0 0x7f68980a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f68ac1981e0 tx=0x7f689c00a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.688+0000 7f68c0730b80 1 Processor -- start 2024-05-08T22:35:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.688+0000 7f68c0730b80 1 -- start start 2024-05-08T22:35:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.688+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.688+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd6e50 0x5599c9cd72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.688+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd7810 0x5599c9cd7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.688+0000 7f68c0730b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9c2d100 con 0x5599c9cd64f0 2024-05-08T22:35:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.688+0000 7f68c0730b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9c2aac0 con 0x5599c9cd7810 2024-05-08T22:35:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.688+0000 7f68c0730b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9c263b0 con 0x5599c9cd6e50 2024-05-08T22:35:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.688+0000 7f68b2312700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.688+0000 7f68b2312700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44352/0 (socket says 172.21.15.62:44352) 2024-05-08T22:35:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68b2312700 1 -- 172.21.15.62:0/1286484134 learned_addr learned my addr 172.21.15.62:0/1286484134 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68b2b13700 1 --2- 172.21.15.62:0/1286484134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd7810 0x5599c9cd7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68b1b11700 1 --2- 172.21.15.62:0/1286484134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd6e50 0x5599c9cd72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68b2312700 1 -- 172.21.15.62:0/1286484134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd6e50 msgr2=0x5599c9cd72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68b2312700 1 --2- 172.21.15.62:0/1286484134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd6e50 0x5599c9cd72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68b2312700 1 -- 172.21.15.62:0/1286484134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd7810 msgr2=0x5599c9cd7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68b2312700 1 --2- 172.21.15.62:0/1286484134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd7810 0x5599c9cd7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68b2312700 1 -- 172.21.15.62:0/1286484134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c9aa7360 con 0x5599c9cd64f0 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68b1b11700 1 --2- 172.21.15.62:0/1286484134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd6e50 0x5599c9cd72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68b2312700 1 --2- 172.21.15.62:0/1286484134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd6910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f689c00fed0 tx=0x7f689c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bb511650a6646b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68a0ff9700 1 -- 172.21.15.62:0/1286484134 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c0108c0 con 0x5599c9cd64f0 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68a0ff9700 1 -- 172.21.15.62:0/1286484134 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f689c020070 con 0x5599c9cd64f0 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.689+0000 7f68a0ff9700 1 -- 172.21.15.62:0/1286484134 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c01be70 con 0x5599c9cd64f0 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.690+0000 7f68c0730b80 1 -- 172.21.15.62:0/1286484134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 msgr2=0x5599c9cd6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.690+0000 7f68c0730b80 1 --2- 172.21.15.62:0/1286484134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd6910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f689c00fed0 tx=0x7f689c012000 comp rx=0 tx=0).stop 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.690+0000 7f68c0730b80 1 -- 172.21.15.62:0/1286484134 shutdown_connections 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.690+0000 7f68c0730b80 1 --2- 172.21.15.62:0/1286484134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd7810 0x5599c9cd7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.690+0000 7f68c0730b80 1 --2- 172.21.15.62:0/1286484134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd6910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.690+0000 7f68c0730b80 1 --2- 172.21.15.62:0/1286484134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd6e50 0x5599c9cd72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.690+0000 7f68c0730b80 1 -- 172.21.15.62:0/1286484134 >> 172.21.15.62:0/1286484134 conn(0x5599c9c341c0 msgr2=0x5599c9cd0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.690+0000 7f68c0730b80 1 -- 172.21.15.62:0/1286484134 shutdown_connections 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.690+0000 7f68c0730b80 1 -- 172.21.15.62:0/1286484134 wait complete. 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.690+0000 7f68c0730b80 1 -- start start 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.690+0000 7f68c0730b80 1 Processor -- start 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cdfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd6e50 0x5599c9ce0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd7810 0x5599c9ce7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68c0730b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9bc2510 con 0x5599c9cd64f0 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68c0730b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9bc21b0 con 0x5599c9cd6e50 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68c0730b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9aa9650 con 0x5599c9cd7810 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68b2312700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cdfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68c0730b80 10 client.0 _ll_register_callbacks cb 0x5599c9bcbae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68b2312700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cdfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44354/0 (socket says 172.21.15.62:44354) 2024-05-08T22:35:55.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68b2312700 1 -- 172.21.15.62:0/2129001954 learned_addr learned my addr 172.21.15.62:0/2129001954 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68b2312700 1 -- 172.21.15.62:0/2129001954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd7810 msgr2=0x5599c9ce7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68b2312700 1 --2- 172.21.15.62:0/2129001954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd7810 0x5599c9ce7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68b2312700 1 -- 172.21.15.62:0/2129001954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd6e50 msgr2=0x5599c9ce0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68b2312700 1 --2- 172.21.15.62:0/2129001954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd6e50 0x5599c9ce0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68b2312700 1 -- 172.21.15.62:0/2129001954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689c00fc10 con 0x5599c9cd64f0 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68b2312700 1 --2- 172.21.15.62:0/2129001954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cdfde0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f689c01be50 tx=0x7f689c01bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3735158e9f60e29e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f6895ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f6895ffb700 1 -- 172.21.15.62:0/2129001954 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c02a070 con 0x5599c9cd64f0 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f6895ffb700 1 -- 172.21.15.62:0/2129001954 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f689c010ad0 con 0x5599c9cd64f0 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68c0730b80 1 -- 172.21.15.62:0/2129001954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599c9b7f220 con 0x5599c9cd64f0 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f6895ffb700 1 -- 172.21.15.62:0/2129001954 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f689c010c90 con 0x5599c9cd64f0 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68c0730b80 10 client.15099 fetch_fsmap learned FSMap version 26 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.691+0000 7f68c0730b80 1 -- 172.21.15.62:0/2129001954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5599c9a53b10 con 0x5599c9cd64f0 2024-05-08T22:35:55.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.692+0000 7f6895ffb700 1 -- 172.21.15.62:0/2129001954 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599c9b7f220 con 0x5599c9cd64f0 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.692+0000 7f6895ffb700 1 -- 172.21.15.62:0/2129001954 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f689c004a80 con 0x5599c9cd64f0 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.692+0000 7f68c0730b80 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.692+0000 7f68c0730b80 10 client.15099 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.692+0000 7f68c0730b80 20 client.15099 populate_metadata read hostname 'smithi062' 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.692+0000 7f68c0730b80 1 --2- 172.21.15.62:0/2129001954 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x5599c9cea940 0x5599c9cf0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.692+0000 7f68c0730b80 4 client.15099 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.692+0000 7f68c0730b80 1 -- 172.21.15.62:0/2129001954 --> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599c9ca57c0 con 0x5599c9cea940 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.692+0000 7f68b1b11700 1 --2- 172.21.15.62:0/2129001954 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x5599c9cea940 0x5599c9cf0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.693+0000 7f68b1b11700 1 --2- 172.21.15.62:0/2129001954 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x5599c9cea940 0x5599c9cf0270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a736f807f6d28306 server_cookie=77f58d5bd452b3e3 in_seq=0 out_seq=0 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.693+0000 7f6895ffb700 10 client.15099 ms_handle_connect on v2:172.21.15.66:6836/1178845415 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.693+0000 7f6895ffb700 1 -- 172.21.15.62:0/2129001954 <== mds.3 v2:172.21.15.66:6836/1178845415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599c9ca57c0 con 0x5599c9cea940 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.693+0000 7f6895ffb700 10 client.15099 handle_command_reply: tid=0 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.693+0000 7f68c0730b80 1 client.15099 shutdown 2024-05-08T22:35:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f68c0730b80 1 -- 172.21.15.62:0/2129001954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 msgr2=0x5599c9cdfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f68c0730b80 1 --2- 172.21.15.62:0/2129001954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cdfde0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f689c01be50 tx=0x7f689c01bf30 comp rx=0 tx=0).stop 2024-05-08T22:35:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f68c0730b80 1 -- 172.21.15.62:0/2129001954 shutdown_connections 2024-05-08T22:35:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f68c0730b80 1 --2- 172.21.15.62:0/2129001954 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x5599c9cea940 0x5599c9cf0270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f68c0730b80 1 --2- 172.21.15.62:0/2129001954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd7810 0x5599c9ce7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f68c0730b80 1 --2- 172.21.15.62:0/2129001954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cdfde0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f6895ffb700 0 client.15099 ms_handle_reset on v2:172.21.15.66:6836/1178845415 2024-05-08T22:35:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f68c0730b80 1 --2- 172.21.15.62:0/2129001954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd6e50 0x5599c9ce0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f68c0730b80 1 -- 172.21.15.62:0/2129001954 >> 172.21.15.62:0/2129001954 conn(0x5599c9c341c0 msgr2=0x5599c9cd0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f68c0730b80 1 -- 172.21.15.62:0/2129001954 shutdown_connections 2024-05-08T22:35:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f68c0730b80 1 -- 172.21.15.62:0/2129001954 wait complete. 2024-05-08T22:35:55.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.694+0000 7f68c0730b80 20 client.15099 trim_cache size 0 max 16384 2024-05-08T22:35:55.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68c0730b80 1 Processor -- start 2024-05-08T22:35:55.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68c0730b80 1 -- start start 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd6e50 0x5599c9cdddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd7810 0x5599c9cd0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68c0730b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9b7f220 con 0x5599c9cd64f0 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68c0730b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9c1f150 con 0x5599c9cd6e50 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68c0730b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9c23010 con 0x5599c9cd7810 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68b2312700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68b2312700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44358/0 (socket says 172.21.15.62:44358) 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68b2312700 1 -- 172.21.15.62:0/4035955990 learned_addr learned my addr 172.21.15.62:0/4035955990 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68b1b11700 1 --2- 172.21.15.62:0/4035955990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd6e50 0x5599c9cdddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68b2312700 1 -- 172.21.15.62:0/4035955990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd7810 msgr2=0x5599c9cd0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68b2312700 1 --2- 172.21.15.62:0/4035955990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd7810 0x5599c9cd0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68b2312700 1 -- 172.21.15.62:0/4035955990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd6e50 msgr2=0x5599c9cdddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68b2312700 1 --2- 172.21.15.62:0/4035955990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd6e50 0x5599c9cdddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68b2312700 1 -- 172.21.15.62:0/4035955990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c9bec510 con 0x5599c9cd64f0 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.712+0000 7f68b1b11700 1 --2- 172.21.15.62:0/4035955990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd6e50 0x5599c9cdddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68b2312700 1 --2- 172.21.15.62:0/4035955990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd17b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f689c01be20 tx=0x7f689c00aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55489caa4f29e6dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68a17fa700 1 -- 172.21.15.62:0/4035955990 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c02a070 con 0x5599c9cd64f0 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68a17fa700 1 -- 172.21.15.62:0/4035955990 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f689c014070 con 0x5599c9cd64f0 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68a17fa700 1 -- 172.21.15.62:0/4035955990 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c004ae0 con 0x5599c9cd64f0 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68c0730b80 1 -- 172.21.15.62:0/4035955990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 msgr2=0x5599c9cd17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68c0730b80 1 --2- 172.21.15.62:0/4035955990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd17b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f689c01be20 tx=0x7f689c00aaf0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68c0730b80 1 -- 172.21.15.62:0/4035955990 shutdown_connections 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68c0730b80 1 --2- 172.21.15.62:0/4035955990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd7810 0x5599c9cd0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68c0730b80 1 --2- 172.21.15.62:0/4035955990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd64f0 0x5599c9cd17b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68c0730b80 1 --2- 172.21.15.62:0/4035955990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd6e50 0x5599c9cdddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68c0730b80 1 -- 172.21.15.62:0/4035955990 >> 172.21.15.62:0/4035955990 conn(0x5599c9c341c0 msgr2=0x5599c9cd7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68c0730b80 1 -- 172.21.15.62:0/4035955990 shutdown_connections 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.713+0000 7f68c0730b80 1 -- 172.21.15.62:0/4035955990 wait complete. 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68c0730b80 1 -- start start 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68c0730b80 1 Processor -- start 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd64f0 0x5599c9ce16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd6e50 0x5599c9ce1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68c0730b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd7810 0x5599c9ce2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68c0730b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9ce3180 con 0x5599c9cd6e50 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68c0730b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9d45f50 con 0x5599c9cd7810 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68c0730b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c9d46110 con 0x5599c9cd64f0 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68c0730b80 10 client.0 _ll_register_callbacks cb 0x5599c9bcbae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68b1b11700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd6e50 0x5599c9ce1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68b1b11700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd6e50 0x5599c9ce1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44368/0 (socket says 172.21.15.62:44368) 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68b1b11700 1 -- 172.21.15.62:0/94192915 learned_addr learned my addr 172.21.15.62:0/94192915 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68b2312700 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd64f0 0x5599c9ce16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.714+0000 7f68b2b13700 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd7810 0x5599c9ce2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68b1b11700 1 -- 172.21.15.62:0/94192915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd64f0 msgr2=0x5599c9ce16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68b1b11700 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd64f0 0x5599c9ce16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68b1b11700 1 -- 172.21.15.62:0/94192915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd7810 msgr2=0x5599c9ce2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68b1b11700 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd7810 0x5599c9ce2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68b1b11700 1 -- 172.21.15.62:0/94192915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599c9a53b10 con 0x5599c9cd6e50 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68b1b11700 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd6e50 0x5599c9ce1c10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f68a80098e0 tx=0x7f68a80094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f642a42949544b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f6895ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f6895ffb700 1 -- 172.21.15.62:0/94192915 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a801d040 con 0x5599c9cd6e50 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f6895ffb700 1 -- 172.21.15.62:0/94192915 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f68a8026410 con 0x5599c9cd6e50 2024-05-08T22:35:55.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68c0730b80 1 -- 172.21.15.62:0/94192915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f689c020070 con 0x5599c9cd6e50 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f6895ffb700 1 -- 172.21.15.62:0/94192915 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a80265d0 con 0x5599c9cd6e50 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68c0730b80 10 client.15105 fetch_fsmap learned FSMap version 26 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68c0730b80 1 -- 172.21.15.62:0/94192915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5599c9aabc80 con 0x5599c9cd6e50 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f6895ffb700 1 -- 172.21.15.62:0/94192915 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f689c020070 con 0x5599c9cd6e50 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f6895ffb700 1 -- 172.21.15.62:0/94192915 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f68a80099f0 con 0x5599c9cd6e50 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68c0730b80 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68c0730b80 10 client.15105 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68c0730b80 20 client.15105 populate_metadata read hostname 'smithi062' 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68c0730b80 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x5599c9d4ae50 0x5599c9d477e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68c0730b80 4 client.15105 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.715+0000 7f68c0730b80 1 -- 172.21.15.62:0/94192915 --> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5599c9c1ee70 con 0x5599c9d4ae50 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68b2312700 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x5599c9d4ae50 0x5599c9d477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f6895ffb700 10 client.15105 ms_handle_connect on v2:172.21.15.66:6836/1178845415 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68b2312700 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x5599c9d4ae50 0x5599c9d477e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bec167761bdc5c9d server_cookie=1bbfe291b6689252 in_seq=0 out_seq=0 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f6895ffb700 1 -- 172.21.15.62:0/94192915 <== mds.3 v2:172.21.15.66:6836/1178845415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5599c9c1ee70 con 0x5599c9d4ae50 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f6895ffb700 10 client.15105 handle_command_reply: tid=0 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68c0730b80 1 client.15105 shutdown 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68c0730b80 1 -- 172.21.15.62:0/94192915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd6e50 msgr2=0x5599c9ce1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68c0730b80 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd6e50 0x5599c9ce1c10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f68a80098e0 tx=0x7f68a80094b0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68c0730b80 1 -- 172.21.15.62:0/94192915 shutdown_connections 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68c0730b80 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599c9cd7810 0x5599c9ce2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68c0730b80 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x5599c9d4ae50 0x5599c9d477e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68c0730b80 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599c9cd64f0 0x5599c9ce16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68c0730b80 1 --2- 172.21.15.62:0/94192915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599c9cd6e50 0x5599c9ce1c10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f6895ffb700 0 client.15105 ms_handle_reset on v2:172.21.15.66:6836/1178845415 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68c0730b80 1 -- 172.21.15.62:0/94192915 >> 172.21.15.62:0/94192915 conn(0x5599c9c341c0 msgr2=0x5599c9cddd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68c0730b80 1 -- 172.21.15.62:0/94192915 shutdown_connections 2024-05-08T22:35:55.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.716+0000 7f68c0730b80 1 -- 172.21.15.62:0/94192915 wait complete. 2024-05-08T22:35:55.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68c0730b80 20 client.15105 trim_cache size 0 max 16384 2024-05-08T22:35:55.724 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:35:55.724 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:35:55.724 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 3, 2024-05-08T22:35:55.724 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24503, 2024-05-08T22:35:55.724 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-05-08T22:35:55.724 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-05-08T22:35:55.724 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-05-08T22:35:55.724 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 40.870054346000003, 2024-05-08T22:35:55.725 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 26, 2024-05-08T22:35:55.725 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 79, 2024-05-08T22:35:55.725 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T22:35:55.725 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 58.466744876999996 2024-05-08T22:35:55.725 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:35:55.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 -- 172.21.15.62:0/1496743085 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f68980a11b0 msgr2=0x7f68980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f68980a11b0 0x7f68980a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f68ac1981e0 tx=0x7f689c00a3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 -- 172.21.15.62:0/1496743085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac101900 msgr2=0x7f68ac19e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac101900 0x7f68ac19e280 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f68a4007ee0 tx=0x7f68a400d3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 -- 172.21.15.62:0/1496743085 shutdown_connections 2024-05-08T22:35:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac101900 0x7f68ac19e280 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac0ffd70 0x7f68ac196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f68980a11b0 0x7f68980a3670 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 --2- 172.21.15.62:0/1496743085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68ac100f60 0x7f68ac1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 -- 172.21.15.62:0/1496743085 >> 172.21.15.62:0/1496743085 conn(0x7f68ac0fb2a0 msgr2=0x7f68ac0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 -- 172.21.15.62:0/1496743085 shutdown_connections 2024-05-08T22:35:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.717+0000 7f68b4576700 1 -- 172.21.15.62:0/1496743085 wait complete. 2024-05-08T22:35:55.729 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T22:35:55.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.866+0000 7f946382e700 1 -- 172.21.15.62:0/1593518468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f945c101fc0 msgr2=0x7f945c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.866+0000 7f946382e700 1 --2- 172.21.15.62:0/1593518468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f945c101fc0 0x7f945c102440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9458009990 tx=0x7f9458009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.866+0000 7f946382e700 1 -- 172.21.15.62:0/1593518468 shutdown_connections 2024-05-08T22:35:55.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.866+0000 7f946382e700 1 --2- 172.21.15.62:0/1593518468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c102980 0x7f945c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.866+0000 7f946382e700 1 --2- 172.21.15.62:0/1593518468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f945c100e70 0x7f945c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.866+0000 7f946382e700 1 --2- 172.21.15.62:0/1593518468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f945c101fc0 0x7f945c102440 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.866+0000 7f946382e700 1 -- 172.21.15.62:0/1593518468 >> 172.21.15.62:0/1593518468 conn(0x7f945c0fc400 msgr2=0x7f945c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.866+0000 7f946382e700 1 -- 172.21.15.62:0/1593518468 shutdown_connections 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.866+0000 7f946382e700 1 -- 172.21.15.62:0/1593518468 wait complete. 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f946382e700 1 Processor -- start 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f946382e700 1 -- start start 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f946382e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f945c100e70 0x7f945c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f946382e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f945c101fc0 0x7f945c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f946382e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c102980 0x7f945c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f946382e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c074070 con 0x7f945c100e70 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f946382e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c0741e0 con 0x7f945c102980 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f946382e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c1a0dc0 con 0x7f945c101fc0 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f94615ca700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f945c100e70 0x7f945c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f94615ca700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f945c100e70 0x7f945c074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44378/0 (socket says 172.21.15.62:44378) 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f94615ca700 1 -- 172.21.15.62:0/268901898 learned_addr learned my addr 172.21.15.62:0/268901898 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f9461dcb700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c102980 0x7f945c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f9460dc9700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f945c101fc0 0x7f945c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f94615ca700 1 -- 172.21.15.62:0/268901898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f945c101fc0 msgr2=0x7f945c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f94615ca700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f945c101fc0 0x7f945c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f94615ca700 1 -- 172.21.15.62:0/268901898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c102980 msgr2=0x7f945c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f94615ca700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c102980 0x7f945c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.867+0000 7f94615ca700 1 -- 172.21.15.62:0/268901898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94580095f0 con 0x7f945c100e70 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.868+0000 7f9460dc9700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f945c101fc0 0x7f945c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.868+0000 7f94615ca700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f945c100e70 0x7f945c074d00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c9b0 tx=0x7f944c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.868+0000 7f94527fc700 1 -- 172.21.15.62:0/268901898 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c007aa0 con 0x7f945c100e70 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.868+0000 7f94527fc700 1 -- 172.21.15.62:0/268901898 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f944c007c00 con 0x7f945c100e70 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.868+0000 7f946382e700 1 -- 172.21.15.62:0/268901898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945c1a0ff0 con 0x7f945c100e70 2024-05-08T22:35:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.868+0000 7f94527fc700 1 -- 172.21.15.62:0/268901898 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c018660 con 0x7f945c100e70 2024-05-08T22:35:55.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.868+0000 7f946382e700 1 -- 172.21.15.62:0/268901898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945c1a1470 con 0x7f945c100e70 2024-05-08T22:35:55.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.871+0000 7f94527fc700 1 -- 172.21.15.62:0/268901898 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f944c00f450 con 0x7f945c100e70 2024-05-08T22:35:55.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.871+0000 7f94527fc700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f94480a11b0 0x7f94480a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.871+0000 7f9460dc9700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f94480a11b0 0x7f94480a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.871+0000 7f94527fc700 1 -- 172.21.15.62:0/268901898 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f944c0cc7e0 con 0x7f945c100e70 2024-05-08T22:35:55.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f9460dc9700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f94480a11b0 0x7f94480a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f945800b2e0 tx=0x7f9458005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f946f9e8b80 1 Processor -- start 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f946f9e8b80 1 -- start start 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8bae50 0x55fd3c8bb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8bbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c811100 con 0x55fd3c8ba4f0 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c80eac0 con 0x55fd3c8bb810 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c80a3b0 con 0x55fd3c8bae50 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f94615ca700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f94615ca700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8ba910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44390/0 (socket says 172.21.15.62:44390) 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f94615ca700 1 -- 172.21.15.62:0/3376115639 learned_addr learned my addr 172.21.15.62:0/3376115639 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f9460dc9700 1 --2- 172.21.15.62:0/3376115639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8bae50 0x55fd3c8bb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.872+0000 7f9461dcb700 1 --2- 172.21.15.62:0/3376115639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8bbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.873+0000 7f94615ca700 1 -- 172.21.15.62:0/3376115639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8bae50 msgr2=0x55fd3c8bb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.873+0000 7f94615ca700 1 --2- 172.21.15.62:0/3376115639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8bae50 0x55fd3c8bb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.873+0000 7f94615ca700 1 -- 172.21.15.62:0/3376115639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 msgr2=0x55fd3c8bbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.873+0000 7f94615ca700 1 --2- 172.21.15.62:0/3376115639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8bbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.873+0000 7f94615ca700 1 -- 172.21.15.62:0/3376115639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd3c68b360 con 0x55fd3c8ba4f0 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.873+0000 7f9460dc9700 1 --2- 172.21.15.62:0/3376115639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8bae50 0x55fd3c8bb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.873+0000 7f9461dcb700 1 --2- 172.21.15.62:0/3376115639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8bbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.873+0000 7f94615ca700 1 --2- 172.21.15.62:0/3376115639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8ba910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f944c0ca390 tx=0x7f944c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5d9b8c4daa9c29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.873+0000 7f94477fe700 1 -- 172.21.15.62:0/3376115639 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c09dab0 con 0x55fd3c8ba4f0 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.873+0000 7f94477fe700 1 -- 172.21.15.62:0/3376115639 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f944c0a2100 con 0x55fd3c8ba4f0 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.873+0000 7f94477fe700 1 -- 172.21.15.62:0/3376115639 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c099ee0 con 0x55fd3c8ba4f0 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.874+0000 7f946f9e8b80 1 -- 172.21.15.62:0/3376115639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8ba4f0 msgr2=0x55fd3c8ba910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.874+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/3376115639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8ba910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f944c0ca390 tx=0x7f944c00cf00 comp rx=0 tx=0).stop 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.874+0000 7f946f9e8b80 1 -- 172.21.15.62:0/3376115639 shutdown_connections 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.874+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/3376115639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8bbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.874+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/3376115639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8ba910 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.874+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/3376115639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8bae50 0x55fd3c8bb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.874+0000 7f946f9e8b80 1 -- 172.21.15.62:0/3376115639 >> 172.21.15.62:0/3376115639 conn(0x55fd3c8181c0 msgr2=0x55fd3c8b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.874+0000 7f946f9e8b80 1 -- 172.21.15.62:0/3376115639 shutdown_connections 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.874+0000 7f946f9e8b80 1 -- 172.21.15.62:0/3376115639 wait complete. 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.874+0000 7f946f9e8b80 1 -- start start 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.874+0000 7f946f9e8b80 1 Processor -- start 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c7a6510 con 0x55fd3c8bae50 2024-05-08T22:35:55.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c7a61b0 con 0x55fd3c8bb810 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c68d650 con 0x55fd3c8ba4f0 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f946f9e8b80 10 client.0 _ll_register_callbacks cb 0x55fd3c7afae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9460dc9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9460dc9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8c42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44402/0 (socket says 172.21.15.62:44402) 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9460dc9700 1 -- 172.21.15.62:0/1948775961 learned_addr learned my addr 172.21.15.62:0/1948775961 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f94615ca700 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9461dcb700 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9460dc9700 1 -- 172.21.15.62:0/1948775961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 msgr2=0x55fd3c8c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9460dc9700 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9460dc9700 1 -- 172.21.15.62:0/1948775961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 msgr2=0x55fd3c8cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9460dc9700 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9460dc9700 1 -- 172.21.15.62:0/1948775961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945c1a1470 con 0x55fd3c8bae50 2024-05-08T22:35:55.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f94615ca700 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9461dcb700 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9460dc9700 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8c42c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9458009990 tx=0x7f9458009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba20ac1485932f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9444ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9444ff9700 1 -- 172.21.15.62:0/1948775961 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945802c070 con 0x55fd3c8bae50 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f9444ff9700 1 -- 172.21.15.62:0/1948775961 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f945800fe40 con 0x55fd3c8bae50 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.875+0000 7f946f9e8b80 1 -- 172.21.15.62:0/1948775961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd3c763220 con 0x55fd3c8bae50 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f9444ff9700 1 -- 172.21.15.62:0/1948775961 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f945801e070 con 0x55fd3c8bae50 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f946f9e8b80 10 client.15117 fetch_fsmap learned FSMap version 26 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f946f9e8b80 1 -- 172.21.15.62:0/1948775961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fd3c637b10 con 0x55fd3c8bae50 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f9444ff9700 1 -- 172.21.15.62:0/1948775961 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd3c763220 con 0x55fd3c8bae50 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f9444ff9700 1 -- 172.21.15.62:0/1948775961 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9458031790 con 0x55fd3c8bae50 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f946f9e8b80 10 client.15117 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f946f9e8b80 10 client.15117 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T22:35:55.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f946f9e8b80 20 client.15117 populate_metadata read hostname 'smithi062' 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x55fd3c8ce410 0x55fd3c8d4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f946f9e8b80 4 client.15117 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f946f9e8b80 1 -- 172.21.15.62:0/1948775961 --> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd3c8897c0 con 0x55fd3c8ce410 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.876+0000 7f94615ca700 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x55fd3c8ce410 0x55fd3c8d4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.877+0000 7f9444ff9700 10 client.15117 ms_handle_connect on v2:172.21.15.79:6836/753714978 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.877+0000 7f94615ca700 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x55fd3c8ce410 0x55fd3c8d4250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19e9dd72a9ec387a server_cookie=7c4ab2d1a71aee71 in_seq=0 out_seq=0 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.877+0000 7f9444ff9700 1 -- 172.21.15.62:0/1948775961 <== mds.2 v2:172.21.15.79:6836/753714978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd3c8897c0 con 0x55fd3c8ce410 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f9444ff9700 10 client.15117 handle_command_reply: tid=0 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 1 client.15117 shutdown 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 1 -- 172.21.15.62:0/1948775961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 msgr2=0x55fd3c8c42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8c42c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9458009990 tx=0x7f9458009310 comp rx=0 tx=0).stop 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 1 -- 172.21.15.62:0/1948775961 shutdown_connections 2024-05-08T22:35:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x55fd3c8ce410 0x55fd3c8d4250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/1948775961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8c42c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 1 -- 172.21.15.62:0/1948775961 >> 172.21.15.62:0/1948775961 conn(0x55fd3c8181c0 msgr2=0x55fd3c8b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f9444ff9700 0 client.15117 ms_handle_reset on v2:172.21.15.79:6836/753714978 2024-05-08T22:35:55.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 1 -- 172.21.15.62:0/1948775961 shutdown_connections 2024-05-08T22:35:55.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 1 -- 172.21.15.62:0/1948775961 wait complete. 2024-05-08T22:35:55.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.878+0000 7f946f9e8b80 20 client.15117 trim_cache size 0 max 16384 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.896+0000 7f946f9e8b80 1 Processor -- start 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.896+0000 7f946f9e8b80 1 -- start start 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8ba8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c7c82b0 con 0x55fd3c8bae50 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c803150 con 0x55fd3c8bb810 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c7a95a0 con 0x55fd3c8ba4f0 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9460dc9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9460dc9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44404/0 (socket says 172.21.15.62:44404) 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9460dc9700 1 -- 172.21.15.62:0/3453375446 learned_addr learned my addr 172.21.15.62:0/3453375446 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9461dcb700 1 --2- 172.21.15.62:0/3453375446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9460dc9700 1 -- 172.21.15.62:0/3453375446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 msgr2=0x55fd3c8ba8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f94615ca700 1 --2- 172.21.15.62:0/3453375446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8ba8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9460dc9700 1 --2- 172.21.15.62:0/3453375446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8ba8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9460dc9700 1 -- 172.21.15.62:0/3453375446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 msgr2=0x55fd3c8cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9460dc9700 1 --2- 172.21.15.62:0/3453375446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9460dc9700 1 -- 172.21.15.62:0/3453375446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd3c7d0510 con 0x55fd3c8bae50 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9461dcb700 1 --2- 172.21.15.62:0/3453375446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9460dc9700 1 --2- 172.21.15.62:0/3453375446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8cb9b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9458024420 tx=0x7f945800b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e3f590188e6947 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9447fff700 1 -- 172.21.15.62:0/3453375446 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945802c040 con 0x55fd3c8bae50 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9447fff700 1 -- 172.21.15.62:0/3453375446 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9458005030 con 0x55fd3c8bae50 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.897+0000 7f9447fff700 1 -- 172.21.15.62:0/3453375446 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458026990 con 0x55fd3c8bae50 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.898+0000 7f946f9e8b80 1 -- 172.21.15.62:0/3453375446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 msgr2=0x55fd3c8cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.898+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/3453375446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8cb9b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9458024420 tx=0x7f945800b710 comp rx=0 tx=0).stop 2024-05-08T22:35:55.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.898+0000 7f946f9e8b80 1 -- 172.21.15.62:0/3453375446 shutdown_connections 2024-05-08T22:35:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.898+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/3453375446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bb810 0x55fd3c8cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.898+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/3453375446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8ba8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.898+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/3453375446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bae50 0x55fd3c8cb9b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.898+0000 7f946f9e8b80 1 -- 172.21.15.62:0/3453375446 >> 172.21.15.62:0/3453375446 conn(0x55fd3c8181c0 msgr2=0x55fd3c8b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.898+0000 7f946f9e8b80 1 -- 172.21.15.62:0/3453375446 shutdown_connections 2024-05-08T22:35:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.898+0000 7f946f9e8b80 1 -- 172.21.15.62:0/3453375446 wait complete. 2024-05-08T22:35:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.898+0000 7f946f9e8b80 1 -- start start 2024-05-08T22:35:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.898+0000 7f946f9e8b80 1 Processor -- start 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bae50 0x55fd3c8ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f946f9e8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bb810 0x55fd3c8ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c8cae60 con 0x55fd3c8bb810 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c8cafd0 con 0x55fd3c8bae50 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f946f9e8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3c8cb190 con 0x55fd3c8ba4f0 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f946f9e8b80 10 client.0 _ll_register_callbacks cb 0x55fd3c7afae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f9461dcb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bb810 0x55fd3c8ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f9461dcb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bb810 0x55fd3c8ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44406/0 (socket says 172.21.15.62:44406) 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f9461dcb700 1 -- 172.21.15.62:0/2627635283 learned_addr learned my addr 172.21.15.62:0/2627635283 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f94615ca700 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f9460dc9700 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bae50 0x55fd3c8ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f9461dcb700 1 -- 172.21.15.62:0/2627635283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 msgr2=0x55fd3c8b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f9461dcb700 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f9461dcb700 1 -- 172.21.15.62:0/2627635283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bae50 msgr2=0x55fd3c8ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f9461dcb700 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bae50 0x55fd3c8ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.899+0000 7f9461dcb700 1 -- 172.21.15.62:0/2627635283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd3c637b10 con 0x55fd3c8bb810 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f9461dcb700 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bb810 0x55fd3c8ca840 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9454007b60 tx=0x7f9454007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40bf32528e2fd91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f9444ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f9444ff9700 1 -- 172.21.15.62:0/2627635283 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945400e040 con 0x55fd3c8bb810 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f9444ff9700 1 -- 172.21.15.62:0/2627635283 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9454014a80 con 0x55fd3c8bb810 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f946f9e8b80 1 -- 172.21.15.62:0/2627635283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f944c0a2100 con 0x55fd3c8bb810 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f9444ff9700 1 -- 172.21.15.62:0/2627635283 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9454014c40 con 0x55fd3c8bb810 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f946f9e8b80 10 client.15123 fetch_fsmap learned FSMap version 26 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f946f9e8b80 1 -- 172.21.15.62:0/2627635283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fd3c68fc80 con 0x55fd3c8bb810 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f9444ff9700 1 -- 172.21.15.62:0/2627635283 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f944c0a2100 con 0x55fd3c8bb810 2024-05-08T22:35:55.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f9444ff9700 1 -- 172.21.15.62:0/2627635283 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9454024420 con 0x55fd3c8bb810 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f946f9e8b80 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f946f9e8b80 10 client.15123 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f946f9e8b80 20 client.15123 populate_metadata read hostname 'smithi062' 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x55fd3c92a680 0x55fd3c927030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f946f9e8b80 4 client.15123 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.900+0000 7f946f9e8b80 1 -- 172.21.15.62:0/2627635283 --> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fd3c802e70 con 0x55fd3c92a680 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f94615ca700 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x55fd3c92a680 0x55fd3c927030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f9444ff9700 10 client.15123 ms_handle_connect on v2:172.21.15.79:6836/753714978 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f94615ca700 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x55fd3c92a680 0x55fd3c927030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5bbc7497f44eef0 server_cookie=2f4a3ce2289df322 in_seq=0 out_seq=0 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f9444ff9700 1 -- 172.21.15.62:0/2627635283 <== mds.2 v2:172.21.15.79:6836/753714978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fd3c802e70 con 0x55fd3c92a680 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f9444ff9700 10 client.15123 handle_command_reply: tid=0 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f946f9e8b80 1 client.15123 shutdown 2024-05-08T22:35:55.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f946f9e8b80 1 -- 172.21.15.62:0/2627635283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bb810 msgr2=0x55fd3c8ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bb810 0x55fd3c8ca840 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9454007b60 tx=0x7f9454007e70 comp rx=0 tx=0).stop 2024-05-08T22:35:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f946f9e8b80 1 -- 172.21.15.62:0/2627635283 shutdown_connections 2024-05-08T22:35:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x55fd3c92a680 0x55fd3c927030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd3c8bb810 0x55fd3c8ca840 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fd3c8ba4f0 0x55fd3c8b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f946f9e8b80 1 --2- 172.21.15.62:0/2627635283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3c8bae50 0x55fd3c8ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f9444ff9700 0 client.15123 ms_handle_reset on v2:172.21.15.79:6836/753714978 2024-05-08T22:35:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f946f9e8b80 1 -- 172.21.15.62:0/2627635283 >> 172.21.15.62:0/2627635283 conn(0x55fd3c8181c0 msgr2=0x55fd3c8c3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f946f9e8b80 1 -- 172.21.15.62:0/2627635283 shutdown_connections 2024-05-08T22:35:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.901+0000 7f946f9e8b80 1 -- 172.21.15.62:0/2627635283 wait complete. 2024-05-08T22:35:55.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946f9e8b80 20 client.15123 trim_cache size 0 max 16384 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24505, 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 42.064859298000002, 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 26, 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 79, 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T22:35:55.911 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 60.369862275000003 2024-05-08T22:35:55.912 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:35:55.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 -- 172.21.15.62:0/268901898 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f94480a11b0 msgr2=0x7f94480a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f94480a11b0 0x7f94480a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f945800b2e0 tx=0x7f9458005e80 comp rx=0 tx=0).stop 2024-05-08T22:35:55.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 -- 172.21.15.62:0/268901898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f945c100e70 msgr2=0x7f945c074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:55.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f945c100e70 0x7f945c074d00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c9b0 tx=0x7f944c00ccc0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 -- 172.21.15.62:0/268901898 shutdown_connections 2024-05-08T22:35:55.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f94480a11b0 0x7f94480a3670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f945c102980 0x7f945c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f945c100e70 0x7f945c074d00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 --2- 172.21.15.62:0/268901898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f945c101fc0 0x7f945c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:55.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 -- 172.21.15.62:0/268901898 >> 172.21.15.62:0/268901898 conn(0x7f945c0fc400 msgr2=0x7f945c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:55.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 -- 172.21.15.62:0/268901898 shutdown_connections 2024-05-08T22:35:55.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:55.902+0000 7f946382e700 1 -- 172.21.15.62:0/268901898 wait complete. 2024-05-08T22:35:55.915 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-08T22:35:56.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.042+0000 7fa353480700 1 -- 172.21.15.62:0/3745615516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c100e70 msgr2=0x7fa34c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.042+0000 7fa353480700 1 --2- 172.21.15.62:0/3745615516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c100e70 0x7fa34c101290 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa348009990 tx=0x7fa348009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.042+0000 7fa353480700 1 -- 172.21.15.62:0/3745615516 shutdown_connections 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.042+0000 7fa353480700 1 --2- 172.21.15.62:0/3745615516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c102980 0x7fa34c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.042+0000 7fa353480700 1 --2- 172.21.15.62:0/3745615516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c100e70 0x7fa34c101290 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.042+0000 7fa353480700 1 --2- 172.21.15.62:0/3745615516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa34c101fc0 0x7fa34c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.042+0000 7fa353480700 1 -- 172.21.15.62:0/3745615516 >> 172.21.15.62:0/3745615516 conn(0x7fa34c0fc400 msgr2=0x7fa34c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.043+0000 7fa353480700 1 -- 172.21.15.62:0/3745615516 shutdown_connections 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.043+0000 7fa353480700 1 -- 172.21.15.62:0/3745615516 wait complete. 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.043+0000 7fa353480700 1 Processor -- start 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.043+0000 7fa353480700 1 -- start start 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.043+0000 7fa353480700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa34c100e70 0x7fa34c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.043+0000 7fa353480700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c101fc0 0x7fa34c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.043+0000 7fa353480700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c102980 0x7fa34c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.043+0000 7fa353480700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c074070 con 0x7fa34c102980 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.043+0000 7fa353480700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c0741e0 con 0x7fa34c101fc0 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.043+0000 7fa353480700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c1a0e40 con 0x7fa34c100e70 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa351a1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c102980 0x7fa34c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa351a1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c102980 0x7fa34c0738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44418/0 (socket says 172.21.15.62:44418) 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa351a1d700 1 -- 172.21.15.62:0/445433472 learned_addr learned my addr 172.21.15.62:0/445433472 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa350a1b700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c101fc0 0x7fa34c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa35121c700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa34c100e70 0x7fa34c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa351a1d700 1 -- 172.21.15.62:0/445433472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa34c100e70 msgr2=0x7fa34c074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa351a1d700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa34c100e70 0x7fa34c074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa351a1d700 1 -- 172.21.15.62:0/445433472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c101fc0 msgr2=0x7fa34c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa351a1d700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c101fc0 0x7fa34c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa351a1d700 1 -- 172.21.15.62:0/445433472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3480095f0 con 0x7fa34c102980 2024-05-08T22:35:56.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa351a1d700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c102980 0x7fa34c0738d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa344007ee0 tx=0x7fa34400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa3427fc700 1 -- 172.21.15.62:0/445433472 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34400dcc0 con 0x7fa34c102980 2024-05-08T22:35:56.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa3427fc700 1 -- 172.21.15.62:0/445433472 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa34400f040 con 0x7fa34c102980 2024-05-08T22:35:56.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa353480700 1 -- 172.21.15.62:0/445433472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34c1a10d0 con 0x7fa34c102980 2024-05-08T22:35:56.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa3427fc700 1 -- 172.21.15.62:0/445433472 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344004830 con 0x7fa34c102980 2024-05-08T22:35:56.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.044+0000 7fa353480700 1 -- 172.21.15.62:0/445433472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34c1a15e0 con 0x7fa34c102980 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.046+0000 7fa3427fc700 1 -- 172.21.15.62:0/445433472 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa344003a00 con 0x7fa34c102980 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.047+0000 7fa3427fc700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3380a1230 0x7fa3380a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.047+0000 7fa3427fc700 1 -- 172.21.15.62:0/445433472 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa3440cc120 con 0x7fa34c102980 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.047+0000 7fa35121c700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3380a1230 0x7fa3380a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.047+0000 7fa35121c700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3380a1230 0x7fa3380a36f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa34800b2e0 tx=0x7fa34800b6f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa35f63ab80 1 Processor -- start 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa35f63ab80 1 -- start start 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c0934f0 0x56403c093910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c0942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c094c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bfea100 con 0x56403c094810 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bfe7ac0 con 0x56403c093e50 2024-05-08T22:35:56.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bfe33b0 con 0x56403c0934f0 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa351a1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c094c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa351a1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c094c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44420/0 (socket says 172.21.15.62:44420) 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa351a1d700 1 -- 172.21.15.62:0/2221813714 learned_addr learned my addr 172.21.15.62:0/2221813714 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa350a1b700 1 --2- 172.21.15.62:0/2221813714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c0942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa35121c700 1 --2- 172.21.15.62:0/2221813714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c0934f0 0x56403c093910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa351a1d700 1 -- 172.21.15.62:0/2221813714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c0934f0 msgr2=0x56403c093910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa351a1d700 1 --2- 172.21.15.62:0/2221813714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c0934f0 0x56403c093910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa351a1d700 1 -- 172.21.15.62:0/2221813714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 msgr2=0x56403c0942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa351a1d700 1 --2- 172.21.15.62:0/2221813714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c0942d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa351a1d700 1 -- 172.21.15.62:0/2221813714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56403be64360 con 0x56403c094810 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa350a1b700 1 --2- 172.21.15.62:0/2221813714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c0942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.048+0000 7fa35121c700 1 --2- 172.21.15.62:0/2221813714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c0934f0 0x56403c093910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa351a1d700 1 --2- 172.21.15.62:0/2221813714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c094c90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa3440a1410 tx=0x7fa344099360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38998c5a6f359f04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa3377fe700 1 -- 172.21.15.62:0/2221813714 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3440cb040 con 0x56403c094810 2024-05-08T22:35:56.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa3377fe700 1 -- 172.21.15.62:0/2221813714 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa34409d120 con 0x56403c094810 2024-05-08T22:35:56.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa3377fe700 1 -- 172.21.15.62:0/2221813714 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3440c9dd0 con 0x56403c094810 2024-05-08T22:35:56.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa35f63ab80 1 -- 172.21.15.62:0/2221813714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 msgr2=0x56403c094c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/2221813714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c094c90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa3440a1410 tx=0x7fa344099360 comp rx=0 tx=0).stop 2024-05-08T22:35:56.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa35f63ab80 1 -- 172.21.15.62:0/2221813714 shutdown_connections 2024-05-08T22:35:56.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/2221813714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c094c90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/2221813714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c0934f0 0x56403c093910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/2221813714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c0942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa35f63ab80 1 -- 172.21.15.62:0/2221813714 >> 172.21.15.62:0/2221813714 conn(0x56403bff11c0 msgr2=0x56403c08dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.049+0000 7fa35f63ab80 1 -- 172.21.15.62:0/2221813714 shutdown_connections 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa35f63ab80 1 -- 172.21.15.62:0/2221813714 wait complete. 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa35f63ab80 1 -- start start 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa35f63ab80 1 Processor -- start 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c0934f0 0x56403c09ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c09d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bf7f510 con 0x56403c094810 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bf7f1b0 con 0x56403c093e50 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403be66650 con 0x56403c0934f0 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa35f63ab80 10 client.0 _ll_register_callbacks cb 0x56403bf88ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa351a1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.050+0000 7fa351a1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44426/0 (socket says 172.21.15.62:44426) 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa351a1d700 1 -- 172.21.15.62:0/4155544403 learned_addr learned my addr 172.21.15.62:0/4155544403 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:56.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa350a1b700 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c09d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa35121c700 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c0934f0 0x56403c09ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa351a1d700 1 -- 172.21.15.62:0/4155544403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c0934f0 msgr2=0x56403c09ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa351a1d700 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c0934f0 0x56403c09ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa351a1d700 1 -- 172.21.15.62:0/4155544403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 msgr2=0x56403c09d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa351a1d700 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c09d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa351a1d700 1 -- 172.21.15.62:0/4155544403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34c1a15e0 con 0x56403c094810 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa351a1d700 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a43a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa344004170 tx=0x7fa3440042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1205311fdda3e763 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa334ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa334ff9700 1 -- 172.21.15.62:0/4155544403 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3440cb070 con 0x56403c094810 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa35f63ab80 1 -- 172.21.15.62:0/4155544403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56403bf3c220 con 0x56403c094810 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa334ff9700 1 -- 172.21.15.62:0/4155544403 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa3440994c0 con 0x56403c094810 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa334ff9700 1 -- 172.21.15.62:0/4155544403 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa344099720 con 0x56403c094810 2024-05-08T22:35:56.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa35f63ab80 10 client.15135 fetch_fsmap learned FSMap version 26 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa35f63ab80 1 -- 172.21.15.62:0/4155544403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56403be10b10 con 0x56403c094810 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.051+0000 7fa334ff9700 1 -- 172.21.15.62:0/4155544403 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56403bf3c220 con 0x56403c094810 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.052+0000 7fa334ff9700 1 -- 172.21.15.62:0/4155544403 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa344082020 con 0x56403c094810 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.052+0000 7fa35f63ab80 10 client.15135 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.052+0000 7fa35f63ab80 10 client.15135 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.052+0000 7fa35f63ab80 20 client.15135 populate_metadata read hostname 'smithi062' 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.052+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x56403c0a7c30 0x56403c0ace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.052+0000 7fa35f63ab80 4 client.15135 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.052+0000 7fa35f63ab80 1 -- 172.21.15.62:0/4155544403 --> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56403c0627c0 con 0x56403c0a7c30 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.052+0000 7fa35121c700 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x56403c0a7c30 0x56403c0ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.052+0000 7fa334ff9700 10 client.15135 ms_handle_connect on v2:172.21.15.79:6838/3534785027 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.052+0000 7fa35121c700 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x56403c0a7c30 0x56403c0ace90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c39e1dec752d341 server_cookie=799c4ad7217badc8 in_seq=0 out_seq=0 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa334ff9700 1 -- 172.21.15.62:0/4155544403 <== mds.1 v2:172.21.15.79:6838/3534785027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56403c0627c0 con 0x56403c0a7c30 2024-05-08T22:35:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa334ff9700 10 client.15135 handle_command_reply: tid=0 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa35f63ab80 1 client.15135 shutdown 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa35f63ab80 1 -- 172.21.15.62:0/4155544403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 msgr2=0x56403c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a43a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa344004170 tx=0x7fa3440042d0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa35f63ab80 1 -- 172.21.15.62:0/4155544403 shutdown_connections 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x56403c0a7c30 0x56403c0ace90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a43a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c0934f0 0x56403c09ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa334ff9700 0 client.15135 ms_handle_reset on v2:172.21.15.79:6838/3534785027 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/4155544403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c09d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa35f63ab80 1 -- 172.21.15.62:0/4155544403 >> 172.21.15.62:0/4155544403 conn(0x56403bff11c0 msgr2=0x56403c08d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.053+0000 7fa35f63ab80 1 -- 172.21.15.62:0/4155544403 shutdown_connections 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.054+0000 7fa35f63ab80 1 -- 172.21.15.62:0/4155544403 wait complete. 2024-05-08T22:35:56.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.054+0000 7fa35f63ab80 20 client.15135 trim_cache size 0 max 16384 2024-05-08T22:35:56.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35f63ab80 1 Processor -- start 2024-05-08T22:35:56.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35f63ab80 1 -- start start 2024-05-08T22:35:56.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c0934f0 0x56403c0938d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c09c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c094810 0x56403c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bfe0040 con 0x56403c0934f0 2024-05-08T22:35:56.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bf825d0 con 0x56403c093e50 2024-05-08T22:35:56.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403c09c990 con 0x56403c094810 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35121c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c0934f0 0x56403c0938d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35121c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c0934f0 0x56403c0938d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44428/0 (socket says 172.21.15.62:44428) 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35121c700 1 -- 172.21.15.62:0/2441055756 learned_addr learned my addr 172.21.15.62:0/2441055756 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa350a1b700 1 --2- 172.21.15.62:0/2441055756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c09c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa351a1d700 1 --2- 172.21.15.62:0/2441055756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c094810 0x56403c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35121c700 1 -- 172.21.15.62:0/2441055756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c094810 msgr2=0x56403c0a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35121c700 1 --2- 172.21.15.62:0/2441055756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c094810 0x56403c0a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35121c700 1 -- 172.21.15.62:0/2441055756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 msgr2=0x56403c09c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35121c700 1 --2- 172.21.15.62:0/2441055756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c09c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35121c700 1 -- 172.21.15.62:0/2441055756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56403bfa9510 con 0x56403c0934f0 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa35121c700 1 --2- 172.21.15.62:0/2441055756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c0934f0 0x56403c0938d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa348000c00 tx=0x7fa348020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0a04b97153bb4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa337fff700 1 -- 172.21.15.62:0/2441055756 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3480342f0 con 0x56403c0934f0 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.072+0000 7fa337fff700 1 -- 172.21.15.62:0/2441055756 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa34802a040 con 0x56403c0934f0 2024-05-08T22:35:56.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.073+0000 7fa337fff700 1 -- 172.21.15.62:0/2441055756 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34800fce0 con 0x56403c0934f0 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.073+0000 7fa35f63ab80 1 -- 172.21.15.62:0/2441055756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c0934f0 msgr2=0x56403c0938d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.073+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/2441055756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c0934f0 0x56403c0938d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa348000c00 tx=0x7fa348020000 comp rx=0 tx=0).stop 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.073+0000 7fa35f63ab80 1 -- 172.21.15.62:0/2441055756 shutdown_connections 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.073+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/2441055756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c094810 0x56403c0a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.073+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/2441055756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c0934f0 0x56403c0938d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.073+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/2441055756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c093e50 0x56403c09c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.073+0000 7fa35f63ab80 1 -- 172.21.15.62:0/2441055756 >> 172.21.15.62:0/2441055756 conn(0x56403bff11c0 msgr2=0x56403c08a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.073+0000 7fa35f63ab80 1 -- 172.21.15.62:0/2441055756 shutdown_connections 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.073+0000 7fa35f63ab80 1 -- 172.21.15.62:0/2441055756 wait complete. 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.073+0000 7fa35f63ab80 1 -- start start 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa35f63ab80 1 Processor -- start 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c0934f0 0x56403c09eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c093e50 0x56403c09f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa35f63ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403c0a4410 con 0x56403c094810 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403c0a4580 con 0x56403c0934f0 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa35f63ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403c0a4740 con 0x56403c093e50 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa35f63ab80 10 client.0 _ll_register_callbacks cb 0x56403bf88ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa351a1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa351a1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44430/0 (socket says 172.21.15.62:44430) 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa351a1d700 1 -- 172.21.15.62:0/3070785450 learned_addr learned my addr 172.21.15.62:0/3070785450 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa350a1b700 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c093e50 0x56403c09f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa35121c700 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c0934f0 0x56403c09eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa351a1d700 1 -- 172.21.15.62:0/3070785450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c093e50 msgr2=0x56403c09f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa351a1d700 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c093e50 0x56403c09f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa351a1d700 1 -- 172.21.15.62:0/3070785450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c0934f0 msgr2=0x56403c09eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa351a1d700 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c0934f0 0x56403c09eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa351a1d700 1 -- 172.21.15.62:0/3070785450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34c109bd0 con 0x56403c094810 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.074+0000 7fa35121c700 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c0934f0 0x56403c09eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa351a1d700 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a3ed0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa344097810 tx=0x7fa344004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792f1e6f24339f48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa334ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa334ff9700 1 -- 172.21.15.62:0/3070785450 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344099920 con 0x56403c094810 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa334ff9700 1 -- 172.21.15.62:0/3070785450 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa34409d2f0 con 0x56403c094810 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa35f63ab80 1 -- 172.21.15.62:0/3070785450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa34409d120 con 0x56403c094810 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa334ff9700 1 -- 172.21.15.62:0/3070785450 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3440a16f0 con 0x56403c094810 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa35f63ab80 10 client.15141 fetch_fsmap learned FSMap version 26 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa35f63ab80 1 -- 172.21.15.62:0/3070785450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56403be68c80 con 0x56403c094810 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa334ff9700 1 -- 172.21.15.62:0/3070785450 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa34409d120 con 0x56403c094810 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa334ff9700 1 -- 172.21.15.62:0/3070785450 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa344082070 con 0x56403c094810 2024-05-08T22:35:56.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa35f63ab80 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa35f63ab80 10 client.15141 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa35f63ab80 20 client.15141 populate_metadata read hostname 'smithi062' 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x56403c103170 0x56403c0ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa35f63ab80 4 client.15141 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.075+0000 7fa35f63ab80 1 -- 172.21.15.62:0/3070785450 --> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56403bfdbe70 con 0x56403c103170 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35121c700 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x56403c103170 0x56403c0ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa334ff9700 10 client.15141 ms_handle_connect on v2:172.21.15.79:6838/3534785027 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35121c700 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x56403c103170 0x56403c0ffb80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ba37c9ca607c0fb server_cookie=f57fa491c8201629 in_seq=0 out_seq=0 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa334ff9700 1 -- 172.21.15.62:0/3070785450 <== mds.1 v2:172.21.15.79:6838/3534785027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56403bfdbe70 con 0x56403c103170 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa334ff9700 10 client.15141 handle_command_reply: tid=0 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35f63ab80 1 client.15141 shutdown 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35f63ab80 1 -- 172.21.15.62:0/3070785450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 msgr2=0x56403c0a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a3ed0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa344097810 tx=0x7fa344004000 comp rx=0 tx=0).stop 2024-05-08T22:35:56.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35f63ab80 1 -- 172.21.15.62:0/3070785450 shutdown_connections 2024-05-08T22:35:56.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x56403c103170 0x56403c0ffb80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403c094810 0x56403c0a3ed0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56403c0934f0 0x56403c09eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa334ff9700 0 client.15141 ms_handle_reset on v2:172.21.15.79:6838/3534785027 2024-05-08T22:35:56.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35f63ab80 1 --2- 172.21.15.62:0/3070785450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403c093e50 0x56403c09f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35f63ab80 1 -- 172.21.15.62:0/3070785450 >> 172.21.15.62:0/3070785450 conn(0x56403bff11c0 msgr2=0x56403c09cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35f63ab80 1 -- 172.21.15.62:0/3070785450 shutdown_connections 2024-05-08T22:35:56.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.076+0000 7fa35f63ab80 1 -- 172.21.15.62:0/3070785450 wait complete. 2024-05-08T22:35:56.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa35f63ab80 20 client.15141 trim_cache size 0 max 16384 2024-05-08T22:35:56.083 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24529, 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 43.247464798000003, 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 26, 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 79, 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 57.202622859999998 2024-05-08T22:35:56.084 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:35:56.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 -- 172.21.15.62:0/445433472 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3380a1230 msgr2=0x7fa3380a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3380a1230 0x7fa3380a36f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa34800b2e0 tx=0x7fa34800b6f0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 -- 172.21.15.62:0/445433472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c102980 msgr2=0x7fa34c0738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c102980 0x7fa34c0738d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa344007ee0 tx=0x7fa34400d3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 -- 172.21.15.62:0/445433472 shutdown_connections 2024-05-08T22:35:56.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c102980 0x7fa34c0738d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa34c100e70 0x7fa34c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3380a1230 0x7fa3380a36f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 --2- 172.21.15.62:0/445433472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c101fc0 0x7fa34c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 -- 172.21.15.62:0/445433472 >> 172.21.15.62:0/445433472 conn(0x7fa34c0fc400 msgr2=0x7fa34c1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 -- 172.21.15.62:0/445433472 shutdown_connections 2024-05-08T22:35:56.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.077+0000 7fa353480700 1 -- 172.21.15.62:0/445433472 wait complete. 2024-05-08T22:35:56.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:55 smithi062 ceph-mon[42571]: pgmap v244: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T22:35:56.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:55 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2893883074' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:56.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:55 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/4141835883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:56.089 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T22:35:56.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.217+0000 7faac3186700 1 -- 172.21.15.62:0/3672501722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faabc100ea0 msgr2=0x7faabc1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.217+0000 7faac3186700 1 --2- 172.21.15.62:0/3672501722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faabc100ea0 0x7faabc1012c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faaac009990 tx=0x7faaac009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.217+0000 7faac3186700 1 -- 172.21.15.62:0/3672501722 shutdown_connections 2024-05-08T22:35:56.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.217+0000 7faac3186700 1 --2- 172.21.15.62:0/3672501722 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faabc1029b0 0x7faabc1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.217+0000 7faac3186700 1 --2- 172.21.15.62:0/3672501722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faabc100ea0 0x7faabc1012c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.217+0000 7faac3186700 1 --2- 172.21.15.62:0/3672501722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc101ff0 0x7faabc102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.217+0000 7faac3186700 1 -- 172.21.15.62:0/3672501722 >> 172.21.15.62:0/3672501722 conn(0x7faabc0fc430 msgr2=0x7faabc0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.217+0000 7faac3186700 1 -- 172.21.15.62:0/3672501722 shutdown_connections 2024-05-08T22:35:56.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.217+0000 7faac3186700 1 -- 172.21.15.62:0/3672501722 wait complete. 2024-05-08T22:35:56.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac3186700 1 Processor -- start 2024-05-08T22:35:56.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac3186700 1 -- start start 2024-05-08T22:35:56.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac3186700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faabc100ea0 0x7faabc196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac3186700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc101ff0 0x7faabc197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac3186700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faabc1029b0 0x7faabc19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac3186700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc19e8f0 con 0x7faabc100ea0 2024-05-08T22:35:56.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac3186700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc19ea60 con 0x7faabc101ff0 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac3186700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc19ec20 con 0x7faabc1029b0 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac0f22700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faabc100ea0 0x7faabc196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac0f22700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faabc100ea0 0x7faabc196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44446/0 (socket says 172.21.15.62:44446) 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac0f22700 1 -- 172.21.15.62:0/1817545297 learned_addr learned my addr 172.21.15.62:0/1817545297 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faabbfff700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc101ff0 0x7faabc197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac1723700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faabc1029b0 0x7faabc19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac0f22700 1 -- 172.21.15.62:0/1817545297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faabc1029b0 msgr2=0x7faabc19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac0f22700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faabc1029b0 0x7faabc19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac0f22700 1 -- 172.21.15.62:0/1817545297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc101ff0 msgr2=0x7faabc197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac0f22700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc101ff0 0x7faabc197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac0f22700 1 -- 172.21.15.62:0/1817545297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac0095f0 con 0x7faabc100ea0 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.218+0000 7faac1723700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faabc1029b0 0x7faabc19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.219+0000 7faac0f22700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faabc100ea0 0x7faabc196e00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faaac00b5c0 tx=0x7faaac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.219+0000 7faab9ffb700 1 -- 172.21.15.62:0/1817545297 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac01d070 con 0x7faabc100ea0 2024-05-08T22:35:56.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.219+0000 7faac3186700 1 -- 172.21.15.62:0/1817545297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faabc19eea0 con 0x7faabc100ea0 2024-05-08T22:35:56.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.219+0000 7faab9ffb700 1 -- 172.21.15.62:0/1817545297 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaac003f60 con 0x7faabc100ea0 2024-05-08T22:35:56.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.219+0000 7faab9ffb700 1 -- 172.21.15.62:0/1817545297 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac022d10 con 0x7faabc100ea0 2024-05-08T22:35:56.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.219+0000 7faac3186700 1 -- 172.21.15.62:0/1817545297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc19f210 con 0x7faabc100ea0 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.221+0000 7faab9ffb700 1 -- 172.21.15.62:0/1817545297 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faaac0040d0 con 0x7faabc100ea0 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.222+0000 7faab9ffb700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faaa40a11b0 0x7faaa40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.222+0000 7faabbfff700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faaa40a11b0 0x7faaa40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.222+0000 7faab9ffb700 1 -- 172.21.15.62:0/1817545297 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faaac0cdc30 con 0x7faabc100ea0 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.222+0000 7faabbfff700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faaa40a11b0 0x7faaa40a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faab0005950 tx=0x7faab000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.222+0000 7faacf340b80 1 Processor -- start 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faacf340b80 1 -- start start 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faacf340b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99ce5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faacf340b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 0x55cb99ce62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faacf340b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce6810 0x55cb99ce6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faacf340b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99c3c100 con 0x55cb99ce6810 2024-05-08T22:35:56.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faacf340b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99c39ac0 con 0x55cb99ce54f0 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faacf340b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99c353b0 con 0x55cb99ce5e50 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faac1723700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce6810 0x55cb99ce6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faac1723700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce6810 0x55cb99ce6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44450/0 (socket says 172.21.15.62:44450) 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faac1723700 1 -- 172.21.15.62:0/491440076 learned_addr learned my addr 172.21.15.62:0/491440076 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faac0f22700 1 --2- 172.21.15.62:0/491440076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99ce5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faabbfff700 1 --2- 172.21.15.62:0/491440076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 0x55cb99ce62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faac1723700 1 -- 172.21.15.62:0/491440076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 msgr2=0x55cb99ce62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faac1723700 1 --2- 172.21.15.62:0/491440076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 0x55cb99ce62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faac1723700 1 -- 172.21.15.62:0/491440076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 msgr2=0x55cb99ce5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faac1723700 1 --2- 172.21.15.62:0/491440076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99ce5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faac1723700 1 -- 172.21.15.62:0/491440076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb99ab6360 con 0x55cb99ce6810 2024-05-08T22:35:56.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faabbfff700 1 --2- 172.21.15.62:0/491440076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 0x55cb99ce62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faac1723700 1 --2- 172.21.15.62:0/491440076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce6810 0x55cb99ce6c90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faab4009d50 tx=0x7faab40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ac712a79c428d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.223+0000 7faaa2ffd700 1 -- 172.21.15.62:0/491440076 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4009ea0 con 0x55cb99ce6810 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.224+0000 7faaa2ffd700 1 -- 172.21.15.62:0/491440076 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab4014ac0 con 0x55cb99ce6810 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.224+0000 7faaa2ffd700 1 -- 172.21.15.62:0/491440076 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4013a60 con 0x55cb99ce6810 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.224+0000 7faacf340b80 1 -- 172.21.15.62:0/491440076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce6810 msgr2=0x55cb99ce6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.224+0000 7faacf340b80 1 --2- 172.21.15.62:0/491440076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce6810 0x55cb99ce6c90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faab4009d50 tx=0x7faab40077e0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.224+0000 7faacf340b80 1 -- 172.21.15.62:0/491440076 shutdown_connections 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.224+0000 7faacf340b80 1 --2- 172.21.15.62:0/491440076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce6810 0x55cb99ce6c90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.224+0000 7faacf340b80 1 --2- 172.21.15.62:0/491440076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99ce5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.224+0000 7faacf340b80 1 --2- 172.21.15.62:0/491440076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 0x55cb99ce62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.224+0000 7faacf340b80 1 -- 172.21.15.62:0/491440076 >> 172.21.15.62:0/491440076 conn(0x55cb99c431c0 msgr2=0x55cb99cdfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.224+0000 7faacf340b80 1 -- 172.21.15.62:0/491440076 shutdown_connections 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.224+0000 7faacf340b80 1 -- 172.21.15.62:0/491440076 wait complete. 2024-05-08T22:35:56.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faacf340b80 1 -- start start 2024-05-08T22:35:56.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faacf340b80 1 Processor -- start 2024-05-08T22:35:56.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faacf340b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99ceee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faacf340b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faacf340b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce6810 0x55cb99cf63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faacf340b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99bd1510 con 0x55cb99ce5e50 2024-05-08T22:35:56.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faacf340b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99bd11b0 con 0x55cb99ce54f0 2024-05-08T22:35:56.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faacf340b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99ab8650 con 0x55cb99ce6810 2024-05-08T22:35:56.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faacf340b80 10 client.0 _ll_register_callbacks cb 0x55cb99bdaae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:56.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faabbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faabbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cef3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44464/0 (socket says 172.21.15.62:44464) 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faabbfff700 1 -- 172.21.15.62:0/927497509 learned_addr learned my addr 172.21.15.62:0/927497509 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faac0f22700 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99ceee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faac1723700 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce6810 0x55cb99cf63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faabbfff700 1 -- 172.21.15.62:0/927497509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce6810 msgr2=0x55cb99cf63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.225+0000 7faabbfff700 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce6810 0x55cb99cf63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faabbfff700 1 -- 172.21.15.62:0/927497509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 msgr2=0x55cb99ceee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faabbfff700 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99ceee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faabbfff700 1 -- 172.21.15.62:0/927497509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab4009980 con 0x55cb99ce5e50 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faac1723700 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce6810 0x55cb99cf63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faac0f22700 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99ceee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faabbfff700 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cef3b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faab0010a90 tx=0x7faab00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516b98a598382a74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faa9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:35:56.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faa9bfff700 1 -- 172.21.15.62:0/927497509 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0004a30 con 0x55cb99ce5e50 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faa9bfff700 1 -- 172.21.15.62:0/927497509 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faab0004b90 con 0x55cb99ce5e50 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faacf340b80 1 -- 172.21.15.62:0/927497509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb99b8e220 con 0x55cb99ce5e50 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faa9bfff700 1 -- 172.21.15.62:0/927497509 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab0004d50 con 0x55cb99ce5e50 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faacf340b80 10 client.15153 fetch_fsmap learned FSMap version 26 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faacf340b80 1 -- 172.21.15.62:0/927497509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cb99a62b10 con 0x55cb99ce5e50 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faa9bfff700 1 -- 172.21.15.62:0/927497509 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb99b8e220 con 0x55cb99ce5e50 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faa9bfff700 1 -- 172.21.15.62:0/927497509 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faab001c9b0 con 0x55cb99ce5e50 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faacf340b80 10 client.15153 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faacf340b80 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faacf340b80 20 client.15153 populate_metadata read hostname 'smithi062' 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faacf340b80 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55cb99cf99f0 0x55cb99cff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faacf340b80 4 client.15153 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:35:56.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.226+0000 7faacf340b80 1 -- 172.21.15.62:0/927497509 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb99cb47c0 con 0x55cb99cf99f0 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.227+0000 7faac0f22700 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55cb99cf99f0 0x55cb99cff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.227+0000 7faa9bfff700 10 client.15153 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.227+0000 7faac0f22700 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55cb99cf99f0 0x55cb99cff2e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba09897ca825257 server_cookie=1347e9bd5f65deb5 in_seq=0 out_seq=0 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faa9bfff700 1 -- 172.21.15.62:0/927497509 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb99cb47c0 con 0x55cb99cf99f0 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faa9bfff700 10 client.15153 handle_command_reply: tid=0 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 1 client.15153 shutdown 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 1 -- 172.21.15.62:0/927497509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 msgr2=0x55cb99cef3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cef3b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faab0010a90 tx=0x7faab00047c0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 1 -- 172.21.15.62:0/927497509 shutdown_connections 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55cb99cf99f0 0x55cb99cff2e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce6810 0x55cb99cf63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99ceee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faa9bfff700 0 client.15153 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:35:56.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 1 --2- 172.21.15.62:0/927497509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cef3b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 1 -- 172.21.15.62:0/927497509 >> 172.21.15.62:0/927497509 conn(0x55cb99c431c0 msgr2=0x55cb99ce42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 1 -- 172.21.15.62:0/927497509 shutdown_connections 2024-05-08T22:35:56.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 1 -- 172.21.15.62:0/927497509 wait complete. 2024-05-08T22:35:56.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.228+0000 7faacf340b80 20 client.15153 trim_cache size 0 max 16384 2024-05-08T22:35:56.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.246+0000 7faacf340b80 1 Processor -- start 2024-05-08T22:35:56.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.246+0000 7faacf340b80 1 -- start start 2024-05-08T22:35:56.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faacf340b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce54f0 0x55cb99ce58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faacf340b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 0x55cb99cf63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faacf340b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce6810 0x55cb99cece60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faacf340b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99bf32b0 con 0x55cb99ce54f0 2024-05-08T22:35:56.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faacf340b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99c2e150 con 0x55cb99ce6810 2024-05-08T22:35:56.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faacf340b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99c32010 con 0x55cb99ce5e50 2024-05-08T22:35:56.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faac0f22700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce54f0 0x55cb99ce58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faac0f22700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce54f0 0x55cb99ce58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44474/0 (socket says 172.21.15.62:44474) 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faac0f22700 1 -- 172.21.15.62:0/3986201596 learned_addr learned my addr 172.21.15.62:0/3986201596 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faac1723700 1 --2- 172.21.15.62:0/3986201596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce6810 0x55cb99cece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faabbfff700 1 --2- 172.21.15.62:0/3986201596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 0x55cb99cf63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faac0f22700 1 -- 172.21.15.62:0/3986201596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 msgr2=0x55cb99cf63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faac0f22700 1 --2- 172.21.15.62:0/3986201596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 0x55cb99cf63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faac0f22700 1 -- 172.21.15.62:0/3986201596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce6810 msgr2=0x55cb99cece60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faac0f22700 1 --2- 172.21.15.62:0/3986201596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce6810 0x55cb99cece60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faac0f22700 1 -- 172.21.15.62:0/3986201596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb99bfb510 con 0x55cb99ce54f0 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faac1723700 1 --2- 172.21.15.62:0/3986201596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce6810 0x55cb99cece60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faabbfff700 1 --2- 172.21.15.62:0/3986201596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 0x55cb99cf63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faac0f22700 1 --2- 172.21.15.62:0/3986201596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce54f0 0x55cb99ce58d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faaac097560 tx=0x7faaac009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22898b9d5e34591b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faaa37fe700 1 -- 172.21.15.62:0/3986201596 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac0aeea0 con 0x55cb99ce54f0 2024-05-08T22:35:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faaa37fe700 1 -- 172.21.15.62:0/3986201596 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaac099260 con 0x55cb99ce54f0 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.247+0000 7faaa37fe700 1 -- 172.21.15.62:0/3986201596 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac09a550 con 0x55cb99ce54f0 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 -- 172.21.15.62:0/3986201596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce54f0 msgr2=0x55cb99ce58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 --2- 172.21.15.62:0/3986201596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce54f0 0x55cb99ce58d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faaac097560 tx=0x7faaac009310 comp rx=0 tx=0).stop 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 -- 172.21.15.62:0/3986201596 shutdown_connections 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 --2- 172.21.15.62:0/3986201596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce6810 0x55cb99cece60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 --2- 172.21.15.62:0/3986201596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce54f0 0x55cb99ce58d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 --2- 172.21.15.62:0/3986201596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce5e50 0x55cb99cf63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 -- 172.21.15.62:0/3986201596 >> 172.21.15.62:0/3986201596 conn(0x55cb99c431c0 msgr2=0x55cb99cdc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 -- 172.21.15.62:0/3986201596 shutdown_connections 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 -- 172.21.15.62:0/3986201596 wait complete. 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 -- start start 2024-05-08T22:35:56.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 Processor -- start 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99cdf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cf5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce6810 0x55cb99cf58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99cf5fe0 con 0x55cb99ce5e50 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99cf6150 con 0x55cb99ce54f0 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb99cf6310 con 0x55cb99ce6810 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.248+0000 7faacf340b80 10 client.0 _ll_register_callbacks cb 0x55cb99bdaae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faabbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cf5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faabbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cf5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44490/0 (socket says 172.21.15.62:44490) 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faabbfff700 1 -- 172.21.15.62:0/504858592 learned_addr learned my addr 172.21.15.62:0/504858592 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faac0f22700 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99cdf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faac1723700 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce6810 0x55cb99cf58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faabbfff700 1 -- 172.21.15.62:0/504858592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce6810 msgr2=0x55cb99cf58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faabbfff700 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce6810 0x55cb99cf58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faabbfff700 1 -- 172.21.15.62:0/504858592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 msgr2=0x55cb99cdf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:35:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faabbfff700 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99cdf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faabbfff700 1 -- 172.21.15.62:0/504858592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac022e70 con 0x55cb99ce5e50 2024-05-08T22:35:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faabbfff700 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cf5350 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faab0010010 tx=0x7faab000aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cecb70cbe4674b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:56.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faa9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faa9bfff700 1 -- 172.21.15.62:0/504858592 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab001c6b0 con 0x55cb99ce5e50 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faa9bfff700 1 -- 172.21.15.62:0/504858592 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faab002c030 con 0x55cb99ce5e50 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faacf340b80 1 -- 172.21.15.62:0/504858592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb99c2de70 con 0x55cb99ce5e50 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faa9bfff700 1 -- 172.21.15.62:0/504858592 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab001c810 con 0x55cb99ce5e50 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faacf340b80 10 client.15159 fetch_fsmap learned FSMap version 26 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faacf340b80 1 -- 172.21.15.62:0/504858592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cb99abac80 con 0x55cb99ce5e50 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.249+0000 7faa9bfff700 1 -- 172.21.15.62:0/504858592 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab0041020 con 0x55cb99ce5e50 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.250+0000 7faa9bfff700 1 -- 172.21.15.62:0/504858592 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faab0005310 con 0x55cb99ce5e50 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.250+0000 7faacf340b80 10 client.15159 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.250+0000 7faacf340b80 10 client.15159 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.250+0000 7faacf340b80 20 client.15159 populate_metadata read hostname 'smithi062' 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.250+0000 7faacf340b80 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55cb99cf99f0 0x55cb99d52120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.250+0000 7faacf340b80 4 client.15159 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:35:56.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.250+0000 7faacf340b80 1 -- 172.21.15.62:0/504858592 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cb99a83ca0 con 0x55cb99cf99f0 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.250+0000 7faac0f22700 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55cb99cf99f0 0x55cb99d52120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.250+0000 7faa9bfff700 10 client.15159 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.250+0000 7faac0f22700 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55cb99cf99f0 0x55cb99d52120 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b19d8f6ff7f489c1 server_cookie=b23f35cf5ed18082 in_seq=0 out_seq=0 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faa9bfff700 1 -- 172.21.15.62:0/504858592 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55cb99a83ca0 con 0x55cb99cf99f0 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faa9bfff700 10 client.15159 handle_command_reply: tid=0 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 1 client.15159 shutdown 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 1 -- 172.21.15.62:0/504858592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 msgr2=0x55cb99cf5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cf5350 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faab0010010 tx=0x7faab000aa70 comp rx=0 tx=0).stop 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 1 -- 172.21.15.62:0/504858592 shutdown_connections 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55cb99cf99f0 0x55cb99d52120 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cb99ce6810 0x55cb99cf58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb99ce54f0 0x55cb99cdf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faa9bfff700 0 client.15159 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:35:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 1 --2- 172.21.15.62:0/504858592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb99ce5e50 0x55cb99cf5350 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 1 -- 172.21.15.62:0/504858592 >> 172.21.15.62:0/504858592 conn(0x55cb99c431c0 msgr2=0x55cb99cee380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 1 -- 172.21.15.62:0/504858592 shutdown_connections 2024-05-08T22:35:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 1 -- 172.21.15.62:0/504858592 wait complete. 2024-05-08T22:35:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faacf340b80 20 client.15159 trim_cache size 0 max 16384 2024-05-08T22:35:56.259 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:35:56.259 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:35:56.259 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-05-08T22:35:56.259 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24533, 2024-05-08T22:35:56.260 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-05-08T22:35:56.260 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-05-08T22:35:56.260 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-05-08T22:35:56.260 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 46.445225485000002, 2024-05-08T22:35:56.260 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 26, 2024-05-08T22:35:56.260 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 79, 2024-05-08T22:35:56.260 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T22:35:56.260 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 52.185680116 2024-05-08T22:35:56.260 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:35:56.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faac3186700 1 -- 172.21.15.62:0/1817545297 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faaa40a11b0 msgr2=0x7faaa40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faac3186700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faaa40a11b0 0x7faaa40a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faab0005950 tx=0x7faab000a300 comp rx=0 tx=0).stop 2024-05-08T22:35:56.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faac3186700 1 -- 172.21.15.62:0/1817545297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faabc100ea0 msgr2=0x7faabc196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:56.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faac3186700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faabc100ea0 0x7faabc196e00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faaac00b5c0 tx=0x7faaac004730 comp rx=0 tx=0).stop 2024-05-08T22:35:56.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faac3186700 1 -- 172.21.15.62:0/1817545297 shutdown_connections 2024-05-08T22:35:56.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faac3186700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faabc1029b0 0x7faabc19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faac3186700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faabc100ea0 0x7faabc196e00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.251+0000 7faac3186700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7faaa40a11b0 0x7faaa40a3670 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.252+0000 7faac3186700 1 --2- 172.21.15.62:0/1817545297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faabc101ff0 0x7faabc197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.252+0000 7faac3186700 1 -- 172.21.15.62:0/1817545297 >> 172.21.15.62:0/1817545297 conn(0x7faabc0fc430 msgr2=0x7faabc105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.252+0000 7faac3186700 1 -- 172.21.15.62:0/1817545297 shutdown_connections 2024-05-08T22:35:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:56.252+0000 7faac3186700 1 -- 172.21.15.62:0/1817545297 wait complete. 2024-05-08T22:35:56.264 INFO:tasks.cephadm:Setup complete, yielding 2024-05-08T22:35:56.264 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T22:35:56.273 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-05-08T22:35:56.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-08T22:35:56.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:55 smithi066 ceph-mon[46830]: pgmap v244: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T22:35:56.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:55 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2893883074' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:56.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:55 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/4141835883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:56.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:55 smithi079 ceph-mon[46738]: pgmap v244: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T22:35:56.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:55 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2893883074' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:35:56.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:55 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/4141835883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:35:56.504 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:57.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.448+0000 7f841f08f700 1 -- 172.21.15.62:0/3382381415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8418101a20 msgr2=0x7f8418108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:57.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.448+0000 7f841f08f700 1 --2- 172.21.15.62:0/3382381415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8418101a20 0x7f8418108340 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f841000b0a0 tx=0x7f841000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:35:57.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.448+0000 7f841f08f700 1 -- 172.21.15.62:0/3382381415 shutdown_connections 2024-05-08T22:35:57.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.448+0000 7f841f08f700 1 --2- 172.21.15.62:0/3382381415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8418101a20 0x7f8418108340 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:57.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.448+0000 7f841f08f700 1 --2- 172.21.15.62:0/3382381415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84180fff00 0x7f8418100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:57.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.448+0000 7f841f08f700 1 --2- 172.21.15.62:0/3382381415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8418101060 0x7f84181014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:57.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.448+0000 7f841f08f700 1 -- 172.21.15.62:0/3382381415 >> 172.21.15.62:0/3382381415 conn(0x7f84180fb4a0 msgr2=0x7f84180fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:57.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.448+0000 7f841f08f700 1 -- 172.21.15.62:0/3382381415 shutdown_connections 2024-05-08T22:35:57.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.448+0000 7f841f08f700 1 -- 172.21.15.62:0/3382381415 wait complete. 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.448+0000 7f841f08f700 1 Processor -- start 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.448+0000 7f841f08f700 1 -- start start 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841f08f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84180fff00 0x7f8418197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841f08f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8418101060 0x7f8418197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841f08f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8418101a20 0x7f841819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841f08f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841819eb60 con 0x7f8418101a20 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841f08f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841819eca0 con 0x7f84180fff00 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841f08f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841819ee30 con 0x7f8418101060 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841d62c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8418101a20 0x7f841819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841d62c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8418101a20 0x7f841819e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47238/0 (socket says 172.21.15.62:47238) 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f8417fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8418101060 0x7f8418197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841d62c700 1 -- 172.21.15.62:0/620822251 learned_addr learned my addr 172.21.15.62:0/620822251 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841ce2b700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84180fff00 0x7f8418197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:57.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841d62c700 1 -- 172.21.15.62:0/620822251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8418101060 msgr2=0x7f8418197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:57.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841d62c700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8418101060 0x7f8418197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:57.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841d62c700 1 -- 172.21.15.62:0/620822251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84180fff00 msgr2=0x7f8418197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:57.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841d62c700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84180fff00 0x7f8418197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:57.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841d62c700 1 -- 172.21.15.62:0/620822251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8410009d00 con 0x7f8418101a20 2024-05-08T22:35:57.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841d62c700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8418101a20 0x7f841819e590 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8410004d00 tx=0x7f8410004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:57.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f8415ffb700 1 -- 172.21.15.62:0/620822251 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841000e040 con 0x7f8418101a20 2024-05-08T22:35:57.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f8415ffb700 1 -- 172.21.15.62:0/620822251 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f841001db50 con 0x7f8418101a20 2024-05-08T22:35:57.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f841f08f700 1 -- 172.21.15.62:0/620822251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841819f0b0 con 0x7f8418101a20 2024-05-08T22:35:57.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.449+0000 7f8415ffb700 1 -- 172.21.15.62:0/620822251 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841001baf0 con 0x7f8418101a20 2024-05-08T22:35:57.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.450+0000 7f841f08f700 1 -- 172.21.15.62:0/620822251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841819f570 con 0x7f8418101a20 2024-05-08T22:35:57.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.451+0000 7f841f08f700 1 -- 172.21.15.62:0/620822251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841804ec60 con 0x7f8418101a20 2024-05-08T22:35:57.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.452+0000 7f8415ffb700 1 -- 172.21.15.62:0/620822251 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f841001dcc0 con 0x7f8418101a20 2024-05-08T22:35:57.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.453+0000 7f8415ffb700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84080a1230 0x7f84080a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:57.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.453+0000 7f841ce2b700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84080a1230 0x7f84080a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:57.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.453+0000 7f8415ffb700 1 -- 172.21.15.62:0/620822251 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f84100cee50 con 0x7f8418101a20 2024-05-08T22:35:57.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.453+0000 7f841ce2b700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84080a1230 0x7f84080a36f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f840c007c60 tx=0x7f840c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:57.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.456+0000 7f8415ffb700 1 -- 172.21.15.62:0/620822251 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f84100986c0 con 0x7f8418101a20 2024-05-08T22:35:57.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.676+0000 7f841f08f700 1 -- 172.21.15.62:0/620822251 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84180fd670 con 0x7f84080a1230 2024-05-08T22:35:57.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.677+0000 7f8415ffb700 1 -- 172.21.15.62:0/620822251 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f84180fd670 con 0x7f84080a1230 2024-05-08T22:35:57.677 INFO:teuthology.orchestra.run.smithi062.stdout:Backend: cephadm 2024-05-08T22:35:57.678 INFO:teuthology.orchestra.run.smithi062.stdout:Available: Yes 2024-05-08T22:35:57.678 INFO:teuthology.orchestra.run.smithi062.stdout:Paused: No 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 -- 172.21.15.62:0/620822251 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84080a1230 msgr2=0x7f84080a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84080a1230 0x7f84080a36f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f840c007c60 tx=0x7f840c0073d0 comp rx=0 tx=0).stop 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 -- 172.21.15.62:0/620822251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8418101a20 msgr2=0x7f841819e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8418101a20 0x7f841819e590 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8410004d00 tx=0x7f8410004d30 comp rx=0 tx=0).stop 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 -- 172.21.15.62:0/620822251 shutdown_connections 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8418101a20 0x7f841819e590 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f84080a1230 0x7f84080a36f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84180fff00 0x7f8418197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 --2- 172.21.15.62:0/620822251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8418101060 0x7f8418197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 -- 172.21.15.62:0/620822251 >> 172.21.15.62:0/620822251 conn(0x7f84180fb4a0 msgr2=0x7f84180fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 -- 172.21.15.62:0/620822251 shutdown_connections 2024-05-08T22:35:57.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:57.680+0000 7f841f08f700 1 -- 172.21.15.62:0/620822251 wait complete. 2024-05-08T22:35:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:57 smithi062 ceph-mon[42571]: pgmap v245: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T22:35:58.028 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-08T22:35:58.259 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:35:58.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:57 smithi066 ceph-mon[46830]: pgmap v245: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T22:35:58.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:57 smithi079 ceph-mon[46738]: pgmap v245: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T22:35:59.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.146+0000 7fdfca524700 1 -- 172.21.15.62:0/3211217664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4101060 msgr2=0x7fdfc41014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:59.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.146+0000 7fdfca524700 1 --2- 172.21.15.62:0/3211217664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4101060 0x7fdfc41014e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009990 tx=0x7fdfb8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:35:59.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.146+0000 7fdfca524700 1 -- 172.21.15.62:0/3211217664 shutdown_connections 2024-05-08T22:35:59.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.147+0000 7fdfca524700 1 --2- 172.21.15.62:0/3211217664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc4101a20 0x7fdfc4108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:59.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.147+0000 7fdfca524700 1 --2- 172.21.15.62:0/3211217664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc40fff00 0x7fdfc4100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:59.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.147+0000 7fdfca524700 1 --2- 172.21.15.62:0/3211217664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4101060 0x7fdfc41014e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.147+0000 7fdfca524700 1 -- 172.21.15.62:0/3211217664 >> 172.21.15.62:0/3211217664 conn(0x7fdfc40fb4a0 msgr2=0x7fdfc40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.147+0000 7fdfca524700 1 -- 172.21.15.62:0/3211217664 shutdown_connections 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.147+0000 7fdfca524700 1 -- 172.21.15.62:0/3211217664 wait complete. 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.147+0000 7fdfca524700 1 Processor -- start 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.147+0000 7fdfca524700 1 -- start start 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfca524700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc40fff00 0x7fdfc4196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfca524700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4101060 0x7fdfc41974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfca524700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc4101a20 0x7fdfc419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfca524700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc419ea20 con 0x7fdfc4101060 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfca524700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc419eb60 con 0x7fdfc40fff00 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfca524700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc419ecf0 con 0x7fdfc4101a20 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfc8ac1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc4101a20 0x7fdfc419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfc8ac1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc4101a20 0x7fdfc419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.62:46772/0 (socket says 172.21.15.62:46772) 2024-05-08T22:35:59.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfc3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc40fff00 0x7fdfc4196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:59.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfc8ac1700 1 -- 172.21.15.62:0/3407908143 learned_addr learned my addr 172.21.15.62:0/3407908143 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:35:59.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfc37fe700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4101060 0x7fdfc41974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:59.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfc3fff700 1 -- 172.21.15.62:0/3407908143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc4101a20 msgr2=0x7fdfc419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:59.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfc3fff700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc4101a20 0x7fdfc419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:59.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfc3fff700 1 -- 172.21.15.62:0/3407908143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4101060 msgr2=0x7fdfc41974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:59.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfc3fff700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4101060 0x7fdfc41974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:59.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfc3fff700 1 -- 172.21.15.62:0/3407908143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb80095f0 con 0x7fdfc40fff00 2024-05-08T22:35:59.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.148+0000 7fdfc3fff700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc40fff00 0x7fdfc4196f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400e9f0 tx=0x7fdfb400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:59.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.149+0000 7fdfc17fa700 1 -- 172.21.15.62:0/3407908143 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb400c9c0 con 0x7fdfc40fff00 2024-05-08T22:35:59.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.149+0000 7fdfca524700 1 -- 172.21.15.62:0/3407908143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc419efd0 con 0x7fdfc40fff00 2024-05-08T22:35:59.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.149+0000 7fdfc17fa700 1 -- 172.21.15.62:0/3407908143 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfb4010430 con 0x7fdfc40fff00 2024-05-08T22:35:59.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.149+0000 7fdfc17fa700 1 -- 172.21.15.62:0/3407908143 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4018680 con 0x7fdfc40fff00 2024-05-08T22:35:59.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.149+0000 7fdfca524700 1 -- 172.21.15.62:0/3407908143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc419f3f0 con 0x7fdfc40fff00 2024-05-08T22:35:59.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.150+0000 7fdfca524700 1 -- 172.21.15.62:0/3407908143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc404ec60 con 0x7fdfc40fff00 2024-05-08T22:35:59.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.151+0000 7fdfc17fa700 1 -- 172.21.15.62:0/3407908143 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdfb4010860 con 0x7fdfc40fff00 2024-05-08T22:35:59.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.151+0000 7fdfc17fa700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdfac0a11b0 0x7fdfac0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:35:59.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.152+0000 7fdfc37fe700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdfac0a11b0 0x7fdfac0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:35:59.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.152+0000 7fdfc17fa700 1 -- 172.21.15.62:0/3407908143 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdfb4014070 con 0x7fdfc40fff00 2024-05-08T22:35:59.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.152+0000 7fdfc37fe700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdfac0a11b0 0x7fdfac0a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800b2e0 tx=0x7fdfb8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:35:59.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.157+0000 7fdfc17fa700 1 -- 172.21.15.62:0/3407908143 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdfb4052d60 con 0x7fdfc40fff00 2024-05-08T22:35:59.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:58 smithi066 ceph-mon[46830]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:59.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.370+0000 7fdfca524700 1 -- 172.21.15.62:0/3407908143 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfc4069060 con 0x7fdfac0a11b0 2024-05-08T22:35:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.383+0000 7fdfc17fa700 1 -- 172.21.15.62:0/3407908143 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fdfc4069060 con 0x7fdfac0a11b0 2024-05-08T22:35:59.384 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-08T22:35:59.384 INFO:teuthology.orchestra.run.smithi062.stdout:mds.a smithi062 running (57s) 52s ago 57s 14.3M - 18.2.2-1322-gd21e4f96 af23193ff691 dde50c6dc6ae 2024-05-08T22:35:59.384 INFO:teuthology.orchestra.run.smithi062.stdout:mds.b smithi066 running (55s) 52s ago 55s 14.2M - 18.2.2-1322-gd21e4f96 af23193ff691 9099b83c7d1f 2024-05-08T22:35:59.384 INFO:teuthology.orchestra.run.smithi062.stdout:mds.c smithi079 running (64s) 52s ago 63s 14.2M - 18.2.2-1322-gd21e4f96 af23193ff691 020c8a8911c2 2024-05-08T22:35:59.384 INFO:teuthology.orchestra.run.smithi062.stdout:mds.d smithi062 running (69s) 52s ago 69s 15.7M - 18.2.2-1322-gd21e4f96 af23193ff691 881a253d9902 2024-05-08T22:35:59.384 INFO:teuthology.orchestra.run.smithi062.stdout:mds.e smithi066 running (67s) 52s ago 67s 15.7M - 18.2.2-1322-gd21e4f96 af23193ff691 fd2b386bad2c 2024-05-08T22:35:59.384 INFO:teuthology.orchestra.run.smithi062.stdout:mds.f smithi079 running (65s) 52s ago 65s 16.8M - 18.2.2-1322-gd21e4f96 af23193ff691 23c14ee32dc5 2024-05-08T22:35:59.384 INFO:teuthology.orchestra.run.smithi062.stdout:mds.g smithi062 running (59s) 52s ago 58s 16.0M - 18.2.2-1322-gd21e4f96 af23193ff691 80e9b39879e2 2024-05-08T22:35:59.384 INFO:teuthology.orchestra.run.smithi062.stdout:mds.h smithi066 running (62s) 52s ago 62s 15.8M - 18.2.2-1322-gd21e4f96 af23193ff691 052f79d63a14 2024-05-08T22:35:59.384 INFO:teuthology.orchestra.run.smithi062.stdout:mds.i smithi079 running (60s) 52s ago 60s 15.6M - 18.2.2-1322-gd21e4f96 af23193ff691 c064756bb18e 2024-05-08T22:35:59.384 INFO:teuthology.orchestra.run.smithi062.stdout:mds.j smithi062 running (71s) 52s ago 70s 14.2M - 18.2.2-1322-gd21e4f96 af23193ff691 1df86215d157 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:mds.k smithi066 running (73s) 52s ago 72s 15.8M - 18.2.2-1322-gd21e4f96 af23193ff691 7ef6f6162a4b 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:mds.l smithi079 running (74s) 52s ago 74s 15.8M - 18.2.2-1322-gd21e4f96 af23193ff691 f3df24c9f3a0 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi062 *:9283,8765 running (6m) 52s ago 6m 525M - 18.2.2-1322-gd21e4f96 af23193ff691 948a7fb5e2f1 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi066 *:8443,8765 running (5m) 52s ago 5m 460M - 18.2.2-1322-gd21e4f96 af23193ff691 ad444d37b03f 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.z smithi079 *:8443,8765 running (5m) 52s ago 5m 460M - 18.2.2-1322-gd21e4f96 af23193ff691 6e76e2e671e5 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (6m) 52s ago 6m 78.2M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 2b83cc9886cd 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi066 running (5m) 52s ago 5m 53.1M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 76f1bfc1bd65 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi079 running (5m) 52s ago 5m 52.4M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 6eebdc2cacc1 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (4m) 52s ago 4m 54.5M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 506ab4fb881e 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi066 running (4m) 52s ago 4m 67.7M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 71218d75399a 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi079 running (4m) 52s ago 4m 63.0M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 7789bfa31c19 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (3m) 52s ago 3m 63.3M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 5b44a83358c5 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi066 running (3m) 52s ago 3m 58.4M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 922bf2877350 2024-05-08T22:35:59.385 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi079 running (3m) 52s ago 3m 54.0M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 20c686885a1a 2024-05-08T22:35:59.386 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi062 running (3m) 52s ago 3m 46.8M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 e54e8517a865 2024-05-08T22:35:59.386 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi066 running (2m) 52s ago 2m 61.7M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 bab0022f7a01 2024-05-08T22:35:59.386 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 smithi079 running (2m) 52s ago 2m 59.4M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 dcd6e1833b65 2024-05-08T22:35:59.386 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 smithi062 running (2m) 52s ago 2m 58.4M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 baf6e51d90b0 2024-05-08T22:35:59.386 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 smithi066 running (110s) 52s ago 110s 59.1M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 3e3ee0bf010d 2024-05-08T22:35:59.386 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 smithi079 running (92s) 52s ago 92s 55.2M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 bfaf3c7fa565 2024-05-08T22:35:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 -- 172.21.15.62:0/3407908143 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdfac0a11b0 msgr2=0x7fdfac0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdfac0a11b0 0x7fdfac0a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800b2e0 tx=0x7fdfb8005fb0 comp rx=0 tx=0).stop 2024-05-08T22:35:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 -- 172.21.15.62:0/3407908143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc40fff00 msgr2=0x7fdfc4196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:35:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc40fff00 0x7fdfc4196f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400e9f0 tx=0x7fdfb400ed00 comp rx=0 tx=0).stop 2024-05-08T22:35:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 -- 172.21.15.62:0/3407908143 shutdown_connections 2024-05-08T22:35:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc4101a20 0x7fdfc419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdfac0a11b0 0x7fdfac0a3670 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdfc40fff00 0x7fdfc4196f80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 --2- 172.21.15.62:0/3407908143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4101060 0x7fdfc41974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:35:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 -- 172.21.15.62:0/3407908143 >> 172.21.15.62:0/3407908143 conn(0x7fdfc40fb4a0 msgr2=0x7fdfc4068950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:35:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 -- 172.21.15.62:0/3407908143 shutdown_connections 2024-05-08T22:35:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:35:59.386+0000 7fdfca524700 1 -- 172.21.15.62:0/3407908143 wait complete. 2024-05-08T22:35:59.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:58 smithi079 ceph-mon[46738]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:59.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:58 smithi062 ceph-mon[42571]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:35:59.755 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-08T22:35:59.986 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:36:00.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:35:59 smithi062 ceph-mon[42571]: pgmap v246: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T22:36:00.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:35:59 smithi066 ceph-mon[46830]: pgmap v246: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T22:36:00.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:35:59 smithi079 ceph-mon[46738]: pgmap v246: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T22:36:00.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.886+0000 7f762a933700 1 -- 172.21.15.62:0/994537513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7624102200 msgr2=0x7f7624102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.886+0000 7f762a933700 1 --2- 172.21.15.62:0/994537513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7624102200 0x7f7624102660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f76180099c0 tx=0x7f7618009cd0 comp rx=0 tx=0).stop 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.886+0000 7f762a933700 1 -- 172.21.15.62:0/994537513 shutdown_connections 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.886+0000 7f762a933700 1 --2- 172.21.15.62:0/994537513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7624102ba0 0x7f7624109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.886+0000 7f762a933700 1 --2- 172.21.15.62:0/994537513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7624101000 0x7f7624101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.886+0000 7f762a933700 1 --2- 172.21.15.62:0/994537513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7624102200 0x7f7624102660 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.886+0000 7f762a933700 1 -- 172.21.15.62:0/994537513 >> 172.21.15.62:0/994537513 conn(0x7f76240fc5a0 msgr2=0x7f76240fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.887+0000 7f762a933700 1 -- 172.21.15.62:0/994537513 shutdown_connections 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.887+0000 7f762a933700 1 -- 172.21.15.62:0/994537513 wait complete. 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.887+0000 7f762a933700 1 Processor -- start 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.887+0000 7f762a933700 1 -- start start 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f762a933700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7624101000 0x7f7624074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f762a933700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7624102200 0x7f7624073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f762a933700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7624102ba0 0x7f76240738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f762a933700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7624074070 con 0x7f7624101000 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f762a933700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76240741e0 con 0x7f7624102200 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f762a933700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76241a0e90 con 0x7f7624102ba0 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f7623fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7624101000 0x7f7624074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f7623fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7624101000 0x7f7624074cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47262/0 (socket says 172.21.15.62:47262) 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f7623fff700 1 -- 172.21.15.62:0/2818080036 learned_addr learned my addr 172.21.15.62:0/2818080036 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f7628ed0700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7624102ba0 0x7f76240738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f76237fe700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7624102200 0x7f7624073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f7623fff700 1 -- 172.21.15.62:0/2818080036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7624102ba0 msgr2=0x7f76240738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f7623fff700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7624102ba0 0x7f76240738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f7623fff700 1 -- 172.21.15.62:0/2818080036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7624102200 msgr2=0x7f7624073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f7623fff700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7624102200 0x7f7624073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f7623fff700 1 -- 172.21.15.62:0/2818080036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76180095f0 con 0x7f7624101000 2024-05-08T22:36:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f76237fe700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7624102200 0x7f7624073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f7623fff700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7624101000 0x7f7624074cf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f761400c990 tx=0x7f761400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f76217fa700 1 -- 172.21.15.62:0/2818080036 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614007b00 con 0x7f7624101000 2024-05-08T22:36:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f762a933700 1 -- 172.21.15.62:0/2818080036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76241a1120 con 0x7f7624101000 2024-05-08T22:36:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f76217fa700 1 -- 172.21.15.62:0/2818080036 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7614007c60 con 0x7f7624101000 2024-05-08T22:36:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f76217fa700 1 -- 172.21.15.62:0/2818080036 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614018660 con 0x7f7624101000 2024-05-08T22:36:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.888+0000 7f762a933700 1 -- 172.21.15.62:0/2818080036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76241a1620 con 0x7f7624101000 2024-05-08T22:36:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.889+0000 7f762a933700 1 -- 172.21.15.62:0/2818080036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76241083c0 con 0x7f7624101000 2024-05-08T22:36:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.891+0000 7f76217fa700 1 -- 172.21.15.62:0/2818080036 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7614004020 con 0x7f7624101000 2024-05-08T22:36:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.892+0000 7f76217fa700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f760c0a11b0 0x7f760c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.892+0000 7f76237fe700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f760c0a11b0 0x7f760c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.892+0000 7f76217fa700 1 -- 172.21.15.62:0/2818080036 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f76140cca00 con 0x7f7624101000 2024-05-08T22:36:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.892+0000 7f76237fe700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f760c0a11b0 0x7f760c0a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7618006010 tx=0x7f7618009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:00.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:00.895+0000 7f76217fa700 1 -- 172.21.15.62:0/2818080036 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7614096270 con 0x7f7624101000 2024-05-08T22:36:01.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.109+0000 7f762a933700 1 -- 172.21.15.62:0/2818080036 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f76240613e0 con 0x7f760c0a11b0 2024-05-08T22:36:01.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.115+0000 7f76217fa700 1 -- 172.21.15.62:0/2818080036 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f76240613e0 con 0x7f760c0a11b0 2024-05-08T22:36:01.115 INFO:teuthology.orchestra.run.smithi062.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-08T22:36:01.115 INFO:teuthology.orchestra.run.smithi062.stdout:mds.all 12/12 54s ago 78s smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-05-08T22:36:01.115 INFO:teuthology.orchestra.run.smithi062.stdout:mgr 3/3 54s ago 5m smithi062=x;smithi066=y;smithi079=z;count:3 2024-05-08T22:36:01.115 INFO:teuthology.orchestra.run.smithi062.stdout:mon 3/3 54s ago 5m smithi062:172.21.15.62=a;smithi066:172.21.15.66=b;smithi079:172.21.15.79=c;count:3 2024-05-08T22:36:01.115 INFO:teuthology.orchestra.run.smithi062.stdout:osd 12 54s ago - 2024-05-08T22:36:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 -- 172.21.15.62:0/2818080036 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f760c0a11b0 msgr2=0x7f760c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:01.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f760c0a11b0 0x7f760c0a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7618006010 tx=0x7f7618009f10 comp rx=0 tx=0).stop 2024-05-08T22:36:01.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 -- 172.21.15.62:0/2818080036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7624101000 msgr2=0x7f7624074cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:01.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7624101000 0x7f7624074cf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f761400c990 tx=0x7f761400cd50 comp rx=0 tx=0).stop 2024-05-08T22:36:01.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 -- 172.21.15.62:0/2818080036 shutdown_connections 2024-05-08T22:36:01.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7624102ba0 0x7f76240738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:01.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7624101000 0x7f7624074cf0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f760c0a11b0 0x7f760c0a3670 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 --2- 172.21.15.62:0/2818080036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7624102200 0x7f7624073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 -- 172.21.15.62:0/2818080036 >> 172.21.15.62:0/2818080036 conn(0x7f76240fc5a0 msgr2=0x7f7624105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 -- 172.21.15.62:0/2818080036 shutdown_connections 2024-05-08T22:36:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:01.117+0000 7f762a933700 1 -- 172.21.15.62:0/2818080036 wait complete. 2024-05-08T22:36:01.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:00 smithi062 ceph-mon[42571]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:01.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:00 smithi066 ceph-mon[46830]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:01.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:00 smithi079 ceph-mon[46738]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:01.487 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-08T22:36:01.719 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:36:02.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:01 smithi062 ceph-mon[42571]: pgmap v247: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T22:36:02.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:01 smithi066 ceph-mon[46830]: pgmap v247: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T22:36:02.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:01 smithi079 ceph-mon[46738]: pgmap v247: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T22:36:02.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.762+0000 7f5ea661c700 1 -- 172.21.15.62:0/643343899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0101060 msgr2=0x7f5ea01014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:02.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.762+0000 7f5ea661c700 1 --2- 172.21.15.62:0/643343899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0101060 0x7f5ea01014e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009990 tx=0x7f5e94009ca0 comp rx=0 tx=0).stop 2024-05-08T22:36:02.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.762+0000 7f5ea661c700 1 -- 172.21.15.62:0/643343899 shutdown_connections 2024-05-08T22:36:02.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.762+0000 7f5ea661c700 1 --2- 172.21.15.62:0/643343899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea0101a20 0x7f5ea0108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:02.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.762+0000 7f5ea661c700 1 --2- 172.21.15.62:0/643343899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea00fff00 0x7f5ea0100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:02.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.762+0000 7f5ea661c700 1 --2- 172.21.15.62:0/643343899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0101060 0x7f5ea01014e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:02.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.762+0000 7f5ea661c700 1 -- 172.21.15.62:0/643343899 >> 172.21.15.62:0/643343899 conn(0x7f5ea00fb4a0 msgr2=0x7f5ea00fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:02.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.762+0000 7f5ea661c700 1 -- 172.21.15.62:0/643343899 shutdown_connections 2024-05-08T22:36:02.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.762+0000 7f5ea661c700 1 -- 172.21.15.62:0/643343899 wait complete. 2024-05-08T22:36:02.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.763+0000 7f5ea661c700 1 Processor -- start 2024-05-08T22:36:02.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.763+0000 7f5ea661c700 1 -- start start 2024-05-08T22:36:02.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.763+0000 7f5ea661c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea00fff00 0x7f5ea0195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:02.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.763+0000 7f5ea661c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea0101060 0x7f5ea01963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:02.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.763+0000 7f5ea661c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0101a20 0x7f5ea019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:02.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.763+0000 7f5ea661c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea019d9b0 con 0x7f5ea0101a20 2024-05-08T22:36:02.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.763+0000 7f5ea661c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea019daf0 con 0x7f5ea0101060 2024-05-08T22:36:02.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.763+0000 7f5ea661c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea019dc80 con 0x7f5ea00fff00 2024-05-08T22:36:02.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.763+0000 7f5ea4bb9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0101a20 0x7f5ea019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5ea4bb9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0101a20 0x7f5ea019d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47278/0 (socket says 172.21.15.62:47278) 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5e9ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea00fff00 0x7f5ea0195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5ea4bb9700 1 -- 172.21.15.62:0/613441627 learned_addr learned my addr 172.21.15.62:0/613441627 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea0101060 0x7f5ea01963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5ea4bb9700 1 -- 172.21.15.62:0/613441627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea00fff00 msgr2=0x7f5ea0195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5ea4bb9700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea00fff00 0x7f5ea0195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5ea4bb9700 1 -- 172.21.15.62:0/613441627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea0101060 msgr2=0x7f5ea01963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5ea4bb9700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea0101060 0x7f5ea01963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5ea4bb9700 1 -- 172.21.15.62:0/613441627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e940095f0 con 0x7f5ea0101a20 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5ea4bb9700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0101a20 0x7f5ea019d3e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800b9a0 tx=0x7f5e9800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5e9d7fa700 1 -- 172.21.15.62:0/613441627 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9800d850 con 0x7f5ea0101a20 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5ea661c700 1 -- 172.21.15.62:0/613441627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea019df60 con 0x7f5ea0101a20 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5e9d7fa700 1 -- 172.21.15.62:0/613441627 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e9800de70 con 0x7f5ea0101a20 2024-05-08T22:36:02.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5e9d7fa700 1 -- 172.21.15.62:0/613441627 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98017400 con 0x7f5ea0101a20 2024-05-08T22:36:02.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.764+0000 7f5ea661c700 1 -- 172.21.15.62:0/613441627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea019e380 con 0x7f5ea0101a20 2024-05-08T22:36:02.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.765+0000 7f5ea661c700 1 -- 172.21.15.62:0/613441627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea0107320 con 0x7f5ea0101a20 2024-05-08T22:36:02.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.766+0000 7f5e9d7fa700 1 -- 172.21.15.62:0/613441627 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5e9800e600 con 0x7f5ea0101a20 2024-05-08T22:36:02.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.767+0000 7f5e9d7fa700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e880a11b0 0x7f5e880a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:02.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.767+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e880a11b0 0x7f5e880a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:02.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.767+0000 7f5e9d7fa700 1 -- 172.21.15.62:0/613441627 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5e98006e40 con 0x7f5ea0101a20 2024-05-08T22:36:02.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.767+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e880a11b0 0x7f5e880a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5e90005950 tx=0x7f5e9000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:02.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.771+0000 7f5e9d7fa700 1 -- 172.21.15.62:0/613441627 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5e98095c90 con 0x7f5ea0101a20 2024-05-08T22:36:02.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.983+0000 7f5ea661c700 1 -- 172.21.15.62:0/613441627 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ea002cf00 con 0x7f5e880a11b0 2024-05-08T22:36:02.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.984+0000 7f5e9d7fa700 1 -- 172.21.15.62:0/613441627 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+187 (secure 0 0 0) 0x7f5ea002cf00 con 0x7f5e880a11b0 2024-05-08T22:36:02.984 INFO:teuthology.orchestra.run.smithi062.stdout:HOST ADDR LABELS STATUS 2024-05-08T22:36:02.984 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 172.21.15.62 2024-05-08T22:36:02.984 INFO:teuthology.orchestra.run.smithi062.stdout:smithi066 172.21.15.66 2024-05-08T22:36:02.984 INFO:teuthology.orchestra.run.smithi062.stdout:smithi079 172.21.15.79 2024-05-08T22:36:02.984 INFO:teuthology.orchestra.run.smithi062.stdout:3 hosts in cluster 2024-05-08T22:36:02.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.986+0000 7f5ea661c700 1 -- 172.21.15.62:0/613441627 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e880a11b0 msgr2=0x7f5e880a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.986+0000 7f5ea661c700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e880a11b0 0x7f5e880a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5e90005950 tx=0x7f5e9000a300 comp rx=0 tx=0).stop 2024-05-08T22:36:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.986+0000 7f5ea661c700 1 -- 172.21.15.62:0/613441627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0101a20 msgr2=0x7f5ea019d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.986+0000 7f5ea661c700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0101a20 0x7f5ea019d3e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800b9a0 tx=0x7f5e9800bcb0 comp rx=0 tx=0).stop 2024-05-08T22:36:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.986+0000 7f5ea661c700 1 -- 172.21.15.62:0/613441627 shutdown_connections 2024-05-08T22:36:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.986+0000 7f5ea661c700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0101a20 0x7f5ea019d3e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.986+0000 7f5ea661c700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ea00fff00 0x7f5ea0195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.986+0000 7f5ea661c700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5e880a11b0 0x7f5e880a3670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.986+0000 7f5ea661c700 1 --2- 172.21.15.62:0/613441627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea0101060 0x7f5ea01963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.986+0000 7f5ea661c700 1 -- 172.21.15.62:0/613441627 >> 172.21.15.62:0/613441627 conn(0x7f5ea00fb4a0 msgr2=0x7f5ea0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.987+0000 7f5ea661c700 1 -- 172.21.15.62:0/613441627 shutdown_connections 2024-05-08T22:36:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:02.987+0000 7f5ea661c700 1 -- 172.21.15.62:0/613441627 wait complete. 2024-05-08T22:36:03.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:02 smithi062 ceph-mon[42571]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:03.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:02 smithi066 ceph-mon[46830]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:03.346 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-08T22:36:03.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:02 smithi079 ceph-mon[46738]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:03.577 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:36:04.317 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:03 smithi062 ceph-mon[42571]: pgmap v248: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-08T22:36:04.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:03 smithi066 ceph-mon[46830]: pgmap v248: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-08T22:36:04.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:03 smithi079 ceph-mon[46738]: pgmap v248: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-08T22:36:04.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.496+0000 7fc0b8c53700 1 -- 172.21.15.62:0/2718087151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b41043c0 msgr2=0x7fc0b41067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:04.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.496+0000 7fc0b8c53700 1 --2- 172.21.15.62:0/2718087151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b41043c0 0x7fc0b41067b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009990 tx=0x7fc0a4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:36:04.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.496+0000 7fc0b8c53700 1 -- 172.21.15.62:0/2718087151 shutdown_connections 2024-05-08T22:36:04.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.496+0000 7fc0b8c53700 1 --2- 172.21.15.62:0/2718087151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b4106cf0 0x7fc0b4109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:04.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.496+0000 7fc0b8c53700 1 --2- 172.21.15.62:0/2718087151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b4101a90 0x7fc0b4103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:04.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.496+0000 7fc0b8c53700 1 --2- 172.21.15.62:0/2718087151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b41043c0 0x7fc0b41067b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:04.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.496+0000 7fc0b8c53700 1 -- 172.21.15.62:0/2718087151 >> 172.21.15.62:0/2718087151 conn(0x7fc0b40fb4a0 msgr2=0x7fc0b40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:04.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.496+0000 7fc0b8c53700 1 -- 172.21.15.62:0/2718087151 shutdown_connections 2024-05-08T22:36:04.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.496+0000 7fc0b8c53700 1 -- 172.21.15.62:0/2718087151 wait complete. 2024-05-08T22:36:04.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b8c53700 1 Processor -- start 2024-05-08T22:36:04.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b8c53700 1 -- start start 2024-05-08T22:36:04.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b8c53700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b4101a90 0x7fc0b41990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:04.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b8c53700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b41043c0 0x7fc0b41995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:04.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b8c53700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b4106cf0 0x7fc0b41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:04.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b8c53700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b41a0be0 con 0x7fc0b4106cf0 2024-05-08T22:36:04.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b8c53700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b41a0d20 con 0x7fc0b4101a90 2024-05-08T22:36:04.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b8c53700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b41a0eb0 con 0x7fc0b41043c0 2024-05-08T22:36:04.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b4106cf0 0x7fc0b41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:04.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b4106cf0 0x7fc0b41a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47290/0 (socket says 172.21.15.62:47290) 2024-05-08T22:36:04.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b2d9d700 1 -- 172.21.15.62:0/1901557994 learned_addr learned my addr 172.21.15.62:0/1901557994 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:04.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b259c700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b4101a90 0x7fc0b41990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:04.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b1d9b700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b41043c0 0x7fc0b41995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:04.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b2d9d700 1 -- 172.21.15.62:0/1901557994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b41043c0 msgr2=0x7fc0b41995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:04.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b2d9d700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b41043c0 0x7fc0b41995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:04.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b2d9d700 1 -- 172.21.15.62:0/1901557994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b4101a90 msgr2=0x7fc0b41990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:04.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b2d9d700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b4101a90 0x7fc0b41990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:04.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b2d9d700 1 -- 172.21.15.62:0/1901557994 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a40095f0 con 0x7fc0b4106cf0 2024-05-08T22:36:04.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b259c700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b4101a90 0x7fc0b41990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:04.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.497+0000 7fc0b1d9b700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b41043c0 0x7fc0b41995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:04.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.498+0000 7fc0b2d9d700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b4106cf0 0x7fc0b41a0610 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac007d70 tx=0x7fc0ac00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:04.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.498+0000 7fc0a37fe700 1 -- 172.21.15.62:0/1901557994 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac00ad00 con 0x7fc0b4106cf0 2024-05-08T22:36:04.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.498+0000 7fc0b8c53700 1 -- 172.21.15.62:0/1901557994 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b41a1190 con 0x7fc0b4106cf0 2024-05-08T22:36:04.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.498+0000 7fc0a37fe700 1 -- 172.21.15.62:0/1901557994 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0ac00ae60 con 0x7fc0b4106cf0 2024-05-08T22:36:04.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.498+0000 7fc0a37fe700 1 -- 172.21.15.62:0/1901557994 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0116f0 con 0x7fc0b4106cf0 2024-05-08T22:36:04.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.498+0000 7fc0b8c53700 1 -- 172.21.15.62:0/1901557994 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b41a1640 con 0x7fc0b4106cf0 2024-05-08T22:36:04.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.499+0000 7fc0b8c53700 1 -- 172.21.15.62:0/1901557994 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b41088f0 con 0x7fc0b4106cf0 2024-05-08T22:36:04.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.500+0000 7fc0a37fe700 1 -- 172.21.15.62:0/1901557994 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc0ac0129e0 con 0x7fc0b4106cf0 2024-05-08T22:36:04.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.500+0000 7fc0a37fe700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc09c0a11b0 0x7fc09c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:04.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.501+0000 7fc0b259c700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc09c0a11b0 0x7fc09c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:04.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.501+0000 7fc0a37fe700 1 -- 172.21.15.62:0/1901557994 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc0ac0ccb90 con 0x7fc0b4106cf0 2024-05-08T22:36:04.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.501+0000 7fc0b259c700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc09c0a11b0 0x7fc09c0a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8005fd0 tx=0x7fc0a8005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:04.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.505+0000 7fc0a37fe700 1 -- 172.21.15.62:0/1901557994 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc0ac096380 con 0x7fc0b4106cf0 2024-05-08T22:36:04.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.716+0000 7fc0b8c53700 1 -- 172.21.15.62:0/1901557994 --> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0b40613e0 con 0x7fc09c0a11b0 2024-05-08T22:36:04.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.717+0000 7fc0a37fe700 1 -- 172.21.15.62:0/1901557994 <== mgr.14150 v2:172.21.15.62:6800/4071146880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fc0b40613e0 con 0x7fc09c0a11b0 2024-05-08T22:36:04.717 INFO:teuthology.orchestra.run.smithi062.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-08T22:36:04.718 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440094400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-08T22:36:04.718 INFO:teuthology.orchestra.run.smithi062.stdout:smithi066 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53430091400BGN 372G No 110s ago Has a FileSystem, LVM detected 2024-05-08T22:36:04.718 INFO:teuthology.orchestra.run.smithi062.stdout:smithi079 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013G400BGN 372G No 93s ago Has a FileSystem, LVM detected 2024-05-08T22:36:04.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.719+0000 7fc0b8c53700 1 -- 172.21.15.62:0/1901557994 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc09c0a11b0 msgr2=0x7fc09c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:04.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.719+0000 7fc0b8c53700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc09c0a11b0 0x7fc09c0a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8005fd0 tx=0x7fc0a8005e20 comp rx=0 tx=0).stop 2024-05-08T22:36:04.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.719+0000 7fc0b8c53700 1 -- 172.21.15.62:0/1901557994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b4106cf0 msgr2=0x7fc0b41a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:04.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.719+0000 7fc0b8c53700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b4106cf0 0x7fc0b41a0610 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac007d70 tx=0x7fc0ac00a340 comp rx=0 tx=0).stop 2024-05-08T22:36:04.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.720+0000 7fc0b8c53700 1 -- 172.21.15.62:0/1901557994 shutdown_connections 2024-05-08T22:36:04.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.720+0000 7fc0b8c53700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b4106cf0 0x7fc0b41a0610 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:04.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.720+0000 7fc0b8c53700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc09c0a11b0 0x7fc09c0a3670 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:04.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.720+0000 7fc0b8c53700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b4101a90 0x7fc0b41990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:04.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.720+0000 7fc0b8c53700 1 --2- 172.21.15.62:0/1901557994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b41043c0 0x7fc0b41995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:04.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.720+0000 7fc0b8c53700 1 -- 172.21.15.62:0/1901557994 >> 172.21.15.62:0/1901557994 conn(0x7fc0b40fb4a0 msgr2=0x7fc0b4101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:04.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.720+0000 7fc0b8c53700 1 -- 172.21.15.62:0/1901557994 shutdown_connections 2024-05-08T22:36:04.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:04.720+0000 7fc0b8c53700 1 -- 172.21.15.62:0/1901557994 wait complete. 2024-05-08T22:36:05.018 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:04 smithi062 ceph-mon[42571]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:05.018 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-08T22:36:05.018 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-08T22:36:05.018 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-08T22:36:05.018 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-08T22:36:05.018 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-05-08T22:36:05.018 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-08T22:36:05.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-08T22:36:05.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:04 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-08T22:36:05.120 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T22:36:05.129 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-05-08T22:36:05.129 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-08T22:36:05.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:04 smithi066 ceph-mon[46830]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:05.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-08T22:36:05.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-08T22:36:05.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-08T22:36:05.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-08T22:36:05.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-05-08T22:36:05.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-08T22:36:05.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-08T22:36:05.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:04 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-08T22:36:05.360 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:36:05.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:04 smithi079 ceph-mon[46738]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:05.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-08T22:36:05.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-08T22:36:05.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-08T22:36:05.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-08T22:36:05.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-05-08T22:36:05.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-08T22:36:05.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-08T22:36:05.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:04 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-08T22:36:05.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:05 smithi062 ceph-mon[42571]: pgmap v249: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T22:36:05.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:05 smithi062 ceph-mon[42571]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:05.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-08T22:36:05.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-08T22:36:05.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-08T22:36:05.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-08T22:36:05.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-05-08T22:36:05.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-08T22:36:05.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-08T22:36:05.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:05 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-08T22:36:05.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:05 smithi062 ceph-mon[42571]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T22:36:06.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:05 smithi066 ceph-mon[46830]: pgmap v249: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T22:36:06.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:05 smithi066 ceph-mon[46830]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:06.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-08T22:36:06.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-08T22:36:06.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-08T22:36:06.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-08T22:36:06.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-05-08T22:36:06.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-08T22:36:06.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-08T22:36:06.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:05 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-08T22:36:06.324 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:05 smithi066 ceph-mon[46830]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T22:36:06.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.329+0000 7f3927256700 1 -- 172.21.15.62:0/3177299297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3920104440 msgr2=0x7f3920106830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.329+0000 7f3927256700 1 --2- 172.21.15.62:0/3177299297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3920104440 0x7f3920106830 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f390c009990 tx=0x7f390c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.329+0000 7f3927256700 1 -- 172.21.15.62:0/3177299297 shutdown_connections 2024-05-08T22:36:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.329+0000 7f3927256700 1 --2- 172.21.15.62:0/3177299297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3920106d70 0x7f3920109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.329+0000 7f3927256700 1 --2- 172.21.15.62:0/3177299297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3920101b10 0x7f3920103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.329+0000 7f3927256700 1 --2- 172.21.15.62:0/3177299297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3920104440 0x7f3920106830 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.329+0000 7f3927256700 1 -- 172.21.15.62:0/3177299297 >> 172.21.15.62:0/3177299297 conn(0x7f39200fb4a0 msgr2=0x7f39200fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.329+0000 7f3927256700 1 -- 172.21.15.62:0/3177299297 shutdown_connections 2024-05-08T22:36:06.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.329+0000 7f3927256700 1 -- 172.21.15.62:0/3177299297 wait complete. 2024-05-08T22:36:06.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.329+0000 7f3927256700 1 Processor -- start 2024-05-08T22:36:06.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.329+0000 7f3927256700 1 -- start start 2024-05-08T22:36:06.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f3927256700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3920101b10 0x7f3920196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f3927256700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3920104440 0x7f3920197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f3927256700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3920106d70 0x7f392019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f3927256700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392019e9d0 con 0x7f3920106d70 2024-05-08T22:36:06.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f3927256700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392019eb10 con 0x7f3920101b10 2024-05-08T22:36:06.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f3927256700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392019eca0 con 0x7f3920104440 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f39257f3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3920106d70 0x7f392019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f39257f3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3920106d70 0x7f392019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47304/0 (socket says 172.21.15.62:47304) 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f39257f3700 1 -- 172.21.15.62:0/1616886833 learned_addr learned my addr 172.21.15.62:0/1616886833 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f3924ff2700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3920101b10 0x7f3920196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f391ffff700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3920104440 0x7f3920197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f39257f3700 1 -- 172.21.15.62:0/1616886833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3920104440 msgr2=0x7f3920197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f39257f3700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3920104440 0x7f3920197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f39257f3700 1 -- 172.21.15.62:0/1616886833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3920101b10 msgr2=0x7f3920196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f39257f3700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3920101b10 0x7f3920196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f39257f3700 1 -- 172.21.15.62:0/1616886833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0095f0 con 0x7f3920106d70 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f39257f3700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3920106d70 0x7f392019e400 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3918007d70 tx=0x7f391800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f391dffb700 1 -- 172.21.15.62:0/1616886833 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391800ac50 con 0x7f3920106d70 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f391dffb700 1 -- 172.21.15.62:0/1616886833 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f391800adb0 con 0x7f3920106d70 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f3927256700 1 -- 172.21.15.62:0/1616886833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392019ef80 con 0x7f3920106d70 2024-05-08T22:36:06.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.330+0000 7f391dffb700 1 -- 172.21.15.62:0/1616886833 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180115e0 con 0x7f3920106d70 2024-05-08T22:36:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.331+0000 7f3927256700 1 -- 172.21.15.62:0/1616886833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392019f3a0 con 0x7f3920106d70 2024-05-08T22:36:06.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.331+0000 7f3927256700 1 -- 172.21.15.62:0/1616886833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39200fd020 con 0x7f3920106d70 2024-05-08T22:36:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.333+0000 7f391dffb700 1 -- 172.21.15.62:0/1616886833 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3918012420 con 0x7f3920106d70 2024-05-08T22:36:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.333+0000 7f391dffb700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39100a11b0 0x7f39100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.333+0000 7f3924ff2700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39100a11b0 0x7f39100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.333+0000 7f391dffb700 1 -- 172.21.15.62:0/1616886833 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f39180ccb10 con 0x7f3920106d70 2024-05-08T22:36:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.334+0000 7f3924ff2700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39100a11b0 0x7f39100a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3914005950 tx=0x7f391400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.338+0000 7f391dffb700 1 -- 172.21.15.62:0/1616886833 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3918096260 con 0x7f3920106d70 2024-05-08T22:36:06.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:05 smithi079 ceph-mon[46738]: pgmap v249: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T22:36:06.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:05 smithi079 ceph-mon[46738]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T22:36:06.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-08T22:36:06.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-08T22:36:06.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-08T22:36:06.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-08T22:36:06.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-05-08T22:36:06.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-08T22:36:06.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-08T22:36:06.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:05 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-08T22:36:06.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:05 smithi079 ceph-mon[46738]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T22:36:06.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.582+0000 7f3927256700 1 -- 172.21.15.62:0/1616886833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f392004ec60 con 0x7f3920106d70 2024-05-08T22:36:06.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.583+0000 7f391dffb700 1 -- 172.21.15.62:0/1616886833 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2913 (secure 0 0 0) 0x7f3918016070 con 0x7f3920106d70 2024-05-08T22:36:06.583 INFO:teuthology.orchestra.run.smithi062.stdout:e26 2024-05-08T22:36:06.583 INFO:teuthology.orchestra.run.smithi062.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T22:36:06.583 INFO:teuthology.orchestra.run.smithi062.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T22:36:06.583 INFO:teuthology.orchestra.run.smithi062.stdout:legacy client fscid: 1 2024-05-08T22:36:06.583 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:36:06.583 INFO:teuthology.orchestra.run.smithi062.stdout:Filesystem 'cephfs' (1) 2024-05-08T22:36:06.583 INFO:teuthology.orchestra.run.smithi062.stdout:fs_name cephfs 2024-05-08T22:36:06.583 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 26 2024-05-08T22:36:06.583 INFO:teuthology.orchestra.run.smithi062.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-08T22:36:06.583 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-05-08T22:35:09.803459+0000 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2024-05-08T22:35:16.261557+0000 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:tableserver 0 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:root 0 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:session_timeout 300 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:session_autoclose 300 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:max_file_size 1099511627776 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:max_xattr_size 65536 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:required_client_features {} 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure 0 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure_osd_epoch 0 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:max_mds 5 2024-05-08T22:36:06.584 INFO:teuthology.orchestra.run.smithi062.stdout:in 0,1,2,3,4 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:up {0=24533,1=24529,2=24505,3=24503,4=14661} 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:failed 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:damaged 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:stopped 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:data_pools [3] 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:metadata_pool 2 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:inline_data disabled 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:balancer 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:bal_rank_mask -1 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:standby_count_wanted 0 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b{0:24533} state up:active seq 3 addr [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.f{0:24493} state up:standby-replay seq 1 addr [v2:172.21.15.79:6834/2938225588,v1:172.21.15.79:6835/2938225588] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.i{1:24529} state up:active seq 5 addr [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.e{1:24479} state up:standby-replay seq 1 addr [v2:172.21.15.66:6834/2449812673,v1:172.21.15.66:6835/2449812673] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.585 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c{2:24505} state up:active seq 6 addr [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.586 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.l{2:24466} state up:standby-replay seq 1 addr [v2:172.21.15.79:6832/1379725080,v1:172.21.15.79:6833/1379725080] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.586 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.h{3:24503} state up:active seq 6 addr [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.586 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.k{3:24455} state up:standby-replay seq 1 addr [v2:172.21.15.66:6832/4065089721,v1:172.21.15.66:6833/4065089721] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.586 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a{4:14661} state up:active seq 5 addr [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.586 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.g{4:14649} state up:standby-replay seq 1 addr [v2:172.21.15.62:6838/1756743060,v1:172.21.15.62:6839/1756743060] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.586 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:36:06.586 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:36:06.586 INFO:teuthology.orchestra.run.smithi062.stdout:Standby daemons: 2024-05-08T22:36:06.586 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:36:06.586 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.j{-1:14586} state up:standby seq 1 addr [v2:172.21.15.62:6834/3584665258,v1:172.21.15.62:6835/3584665258] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.586 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d{-1:14598} state up:standby seq 1 addr [v2:172.21.15.62:6836/2939636235,v1:172.21.15.62:6837/2939636235] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T22:36:06.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 -- 172.21.15.62:0/1616886833 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39100a11b0 msgr2=0x7f39100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:06.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39100a11b0 0x7f39100a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3914005950 tx=0x7f391400b410 comp rx=0 tx=0).stop 2024-05-08T22:36:06.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 -- 172.21.15.62:0/1616886833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3920106d70 msgr2=0x7f392019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:06.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3920106d70 0x7f392019e400 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3918007d70 tx=0x7f391800a340 comp rx=0 tx=0).stop 2024-05-08T22:36:06.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 -- 172.21.15.62:0/1616886833 shutdown_connections 2024-05-08T22:36:06.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3920106d70 0x7f392019e400 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f39100a11b0 0x7f39100a3670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3920101b10 0x7f3920196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 --2- 172.21.15.62:0/1616886833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3920104440 0x7f3920197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:06.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 -- 172.21.15.62:0/1616886833 >> 172.21.15.62:0/1616886833 conn(0x7f39200fb4a0 msgr2=0x7f39201023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:06.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 -- 172.21.15.62:0/1616886833 shutdown_connections 2024-05-08T22:36:06.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:06.585+0000 7f3927256700 1 -- 172.21.15.62:0/1616886833 wait complete. 2024-05-08T22:36:06.588 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-05-08T22:36:07.002 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-08T22:36:07.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:06 smithi062 ceph-mon[42571]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T22:36:07.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:06 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1616886833' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T22:36:07.234 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:36:07.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:06 smithi066 ceph-mon[46830]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T22:36:07.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:06 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1616886833' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T22:36:07.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:06 smithi079 ceph-mon[46738]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T22:36:07.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:06 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1616886833' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T22:36:08.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:07 smithi066 ceph-mon[46830]: pgmap v252: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-08T22:36:08.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:07 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:36:08.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:07 smithi079 ceph-mon[46738]: pgmap v252: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-08T22:36:08.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:07 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:36:08.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:07 smithi062 ceph-mon[42571]: pgmap v252: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-08T22:36:08.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:07 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:36:08.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.488+0000 7f49053c8700 1 -- 172.21.15.62:0/958626288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4900074f50 msgr2=0x7f49000733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.488+0000 7f49053c8700 1 --2- 172.21.15.62:0/958626288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4900074f50 0x7f49000733b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009990 tx=0x7f48f0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.488+0000 7f49053c8700 1 -- 172.21.15.62:0/958626288 shutdown_connections 2024-05-08T22:36:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.488+0000 7f49053c8700 1 --2- 172.21.15.62:0/958626288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4900108f00 0x7f490010b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.488+0000 7f49053c8700 1 --2- 172.21.15.62:0/958626288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4900074f50 0x7f49000733b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.488+0000 7f49053c8700 1 --2- 172.21.15.62:0/958626288 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4900073980 0x7f4900073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.488+0000 7f49053c8700 1 -- 172.21.15.62:0/958626288 >> 172.21.15.62:0/958626288 conn(0x7f49000fc5c0 msgr2=0x7f49000fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.488+0000 7f49053c8700 1 -- 172.21.15.62:0/958626288 shutdown_connections 2024-05-08T22:36:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.488+0000 7f49053c8700 1 -- 172.21.15.62:0/958626288 wait complete. 2024-05-08T22:36:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.488+0000 7f49053c8700 1 Processor -- start 2024-05-08T22:36:08.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f49053c8700 1 -- start start 2024-05-08T22:36:08.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f49053c8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4900073980 0x7f4900199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f49053c8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4900074f50 0x7f4900199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f49053c8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4900108f00 0x7f49001a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f49053c8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001a0c20 con 0x7f4900074f50 2024-05-08T22:36:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f49053c8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001a0d60 con 0x7f4900108f00 2024-05-08T22:36:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f49053c8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001a0ef0 con 0x7f4900073980 2024-05-08T22:36:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f48feffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4900073980 0x7f4900199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f48feffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4900073980 0x7f4900199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.62:35068/0 (socket says 172.21.15.62:35068) 2024-05-08T22:36:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f48feffd700 1 -- 172.21.15.62:0/3072710034 learned_addr learned my addr 172.21.15.62:0/3072710034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f48ff7fe700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4900108f00 0x7f49001a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f48fe7fc700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4900074f50 0x7f4900199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f48feffd700 1 -- 172.21.15.62:0/3072710034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4900108f00 msgr2=0x7f49001a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f48feffd700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4900108f00 0x7f49001a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f48feffd700 1 -- 172.21.15.62:0/3072710034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4900074f50 msgr2=0x7f4900199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f48feffd700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4900074f50 0x7f4900199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.489+0000 7f48feffd700 1 -- 172.21.15.62:0/3072710034 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f00095f0 con 0x7f4900073980 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.490+0000 7f48feffd700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4900073980 0x7f4900199130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48f0004710 tx=0x7f48f0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.490+0000 7f48e7fff700 1 -- 172.21.15.62:0/3072710034 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f001d070 con 0x7f4900073980 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.490+0000 7f48e7fff700 1 -- 172.21.15.62:0/3072710034 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f0021410 con 0x7f4900073980 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.490+0000 7f49053c8700 1 -- 172.21.15.62:0/3072710034 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49001a1170 con 0x7f4900073980 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.490+0000 7f48e7fff700 1 -- 172.21.15.62:0/3072710034 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0017650 con 0x7f4900073980 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.490+0000 7f49053c8700 1 -- 172.21.15.62:0/3072710034 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49001a15e0 con 0x7f4900073980 2024-05-08T22:36:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.491+0000 7f49053c8700 1 -- 172.21.15.62:0/3072710034 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4900107d10 con 0x7f4900073980 2024-05-08T22:36:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.492+0000 7f48e7fff700 1 -- 172.21.15.62:0/3072710034 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f48f0021580 con 0x7f4900073980 2024-05-08T22:36:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.493+0000 7f48e7fff700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48e80a1230 0x7f48e80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:08.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.493+0000 7f48e7fff700 1 -- 172.21.15.62:0/3072710034 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f48f00cdba0 con 0x7f4900073980 2024-05-08T22:36:08.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.493+0000 7f48fe7fc700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48e80a1230 0x7f48e80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:08.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.493+0000 7f48fe7fc700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48e80a1230 0x7f48e80a36f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009c60 tx=0x7f48f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:08.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.498+0000 7f48e7fff700 1 -- 172.21.15.62:0/3072710034 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f48f0097320 con 0x7f4900073980 2024-05-08T22:36:08.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.728+0000 7f49053c8700 1 -- 172.21.15.62:0/3072710034 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f490004ec60 con 0x7f4900073980 2024-05-08T22:36:08.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.729+0000 7f48e7fff700 1 -- 172.21.15.62:0/3072710034 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) v1 ==== 56+0+5259 (secure 0 0 0) 0x7f48f009a940 con 0x7f4900073980 2024-05-08T22:36:08.729 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 81 2024-05-08T22:36:08.729 INFO:teuthology.orchestra.run.smithi062.stdout:fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:36:08.729 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-05-08T22:29:16.808489+0000 2024-05-08T22:36:08.729 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2024-05-08T22:36:05.960694+0000 2024-05-08T22:36:08.729 INFO:teuthology.orchestra.run.smithi062.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T22:36:08.729 INFO:teuthology.orchestra.run.smithi062.stdout:crush_version 26 2024-05-08T22:36:08.729 INFO:teuthology.orchestra.run.smithi062.stdout:full_ratio 0.95 2024-05-08T22:36:08.729 INFO:teuthology.orchestra.run.smithi062.stdout:backfillfull_ratio 0.9 2024-05-08T22:36:08.729 INFO:teuthology.orchestra.run.smithi062.stdout:nearfull_ratio 0.85 2024-05-08T22:36:08.729 INFO:teuthology.orchestra.run.smithi062.stdout:require_min_compat_client luminous 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:min_compat_client luminous 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:require_osd_release reef 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:stretch_mode_enabled false 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:max_osd 12 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6802/2221009645,v1:172.21.15.62:6803/2221009645] [v2:172.21.15.62:6804/2221009645,v1:172.21.15.62:6805/2221009645] exists,up 7242466d-5961-4fd3-8d3e-2996fcccd92e 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6800/1982485054,v1:172.21.15.66:6801/1982485054] [v2:172.21.15.66:6802/1982485054,v1:172.21.15.66:6803/1982485054] exists,up 3e358e76-c55b-4e94-a8e6-727c65c10718 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6800/2543768835,v1:172.21.15.79:6801/2543768835] [v2:172.21.15.79:6802/2543768835,v1:172.21.15.79:6803/2543768835] exists,up 4a8002d5-c2cc-4275-aa59-bb335b2b8740 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 up in weight 1 up_from 24 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6810/2546077230,v1:172.21.15.62:6811/2546077230] [v2:172.21.15.62:6812/2546077230,v1:172.21.15.62:6813/2546077230] exists,up 2cb8e9ce-ae6a-4c93-a888-2e0c831691d9 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 up in weight 1 up_from 31 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6808/2311123866,v1:172.21.15.66:6809/2311123866] [v2:172.21.15.66:6810/2311123866,v1:172.21.15.66:6811/2311123866] exists,up 7d3e6d05-7c0c-4098-8e91-e91fce3173cd 2024-05-08T22:36:08.730 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 up in weight 1 up_from 36 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6808/1027194211,v1:172.21.15.79:6809/1027194211] [v2:172.21.15.79:6810/1027194211,v1:172.21.15.79:6811/1027194211] exists,up e7f834ac-4a24-4111-a581-4276baaa08a8 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 up in weight 1 up_from 43 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6818/1217656145,v1:172.21.15.62:6819/1217656145] [v2:172.21.15.62:6820/1217656145,v1:172.21.15.62:6821/1217656145] exists,up da290a10-42f6-4e73-a6e6-e716af055721 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 up in weight 1 up_from 48 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6816/653963757,v1:172.21.15.66:6817/653963757] [v2:172.21.15.66:6818/653963757,v1:172.21.15.66:6819/653963757] exists,up ee23b28b-603a-4f50-b3f1-2404bef4fa74 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 up in weight 1 up_from 55 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6816/2871988040,v1:172.21.15.79:6817/2871988040] [v2:172.21.15.79:6818/2871988040,v1:172.21.15.79:6819/2871988040] exists,up 5f193ce6-5093-45c8-aee0-2c6e99873111 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 up in weight 1 up_from 62 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6826/1806941351,v1:172.21.15.62:6827/1806941351] [v2:172.21.15.62:6828/1806941351,v1:172.21.15.62:6829/1806941351] exists,up 1811353e-323e-4f18-8c1b-5121e896aa68 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 up in weight 1 up_from 67 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6824/1104097413,v1:172.21.15.66:6825/1104097413] [v2:172.21.15.66:6826/1104097413,v1:172.21.15.66:6827/1104097413] exists,up e1769ba4-8dd8-4fb2-a596-e37d465a58d9 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 up in weight 1 up_from 74 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6824/1561471078,v1:172.21.15.79:6825/1561471078] [v2:172.21.15.79:6826/1561471078,v1:172.21.15.79:6827/1561471078] exists,up 5d7d0579-5d22-46e8-a43b-714675f3dd80 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.b [8,11] 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.f [8,11] 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.14 [2,4] 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.1d [2,4] 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.1d [5,9] 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.23 [4,8] 2024-05-08T22:36:08.731 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.2c [4,9] 2024-05-08T22:36:08.732 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.3c [4,9] 2024-05-08T22:36:08.732 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/766518770 expires 2024-05-09T22:30:03.951072+0000 2024-05-08T22:36:08.732 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/766518770 expires 2024-05-09T22:30:03.951072+0000 2024-05-08T22:36:08.732 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/2477775765 expires 2024-05-09T22:30:03.951072+0000 2024-05-08T22:36:08.732 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/2433771662 expires 2024-05-09T22:30:03.951072+0000 2024-05-08T22:36:08.732 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/1318760241 expires 2024-05-09T22:29:43.058069+0000 2024-05-08T22:36:08.732 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/1247031049 expires 2024-05-09T22:29:43.058069+0000 2024-05-08T22:36:08.732 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/3409257313 expires 2024-05-09T22:29:43.058069+0000 2024-05-08T22:36:08.732 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/3409257313 expires 2024-05-09T22:29:43.058069+0000 2024-05-08T22:36:08.732 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/3020122046 expires 2024-05-09T22:29:43.058069+0000 2024-05-08T22:36:08.732 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/2719135336 expires 2024-05-09T22:30:03.951072+0000 2024-05-08T22:36:08.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 -- 172.21.15.62:0/3072710034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48e80a1230 msgr2=0x7f48e80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48e80a1230 0x7f48e80a36f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009c60 tx=0x7f48f4009260 comp rx=0 tx=0).stop 2024-05-08T22:36:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 -- 172.21.15.62:0/3072710034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4900073980 msgr2=0x7f4900199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4900073980 0x7f4900199130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48f0004710 tx=0x7f48f0009580 comp rx=0 tx=0).stop 2024-05-08T22:36:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 -- 172.21.15.62:0/3072710034 shutdown_connections 2024-05-08T22:36:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f48e80a1230 0x7f48e80a36f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4900108f00 0x7f49001a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4900073980 0x7f4900199130 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 --2- 172.21.15.62:0/3072710034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4900074f50 0x7f4900199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 -- 172.21.15.62:0/3072710034 >> 172.21.15.62:0/3072710034 conn(0x7f49000fc5c0 msgr2=0x7f4900104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 -- 172.21.15.62:0/3072710034 shutdown_connections 2024-05-08T22:36:08.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:08.731+0000 7f49053c8700 1 -- 172.21.15.62:0/3072710034 wait complete. 2024-05-08T22:36:09.055 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-08T22:36:09.070 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:09.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.203+0000 7fe48fe39700 1 -- 172.21.15.62:0/3477942944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4880ffd80 msgr2=0x7fe4881001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.203+0000 7fe48fe39700 1 --2- 172.21.15.62:0/3477942944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4880ffd80 0x7fe4881001a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe4840099c0 tx=0x7fe484009cd0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.203+0000 7fe48fe39700 1 -- 172.21.15.62:0/3477942944 shutdown_connections 2024-05-08T22:36:09.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.203+0000 7fe48fe39700 1 --2- 172.21.15.62:0/3477942944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488101910 0x7fe4881081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.203+0000 7fe48fe39700 1 --2- 172.21.15.62:0/3477942944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4880ffd80 0x7fe4881001a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.203+0000 7fe48fe39700 1 --2- 172.21.15.62:0/3477942944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe488100f70 0x7fe4881013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.203+0000 7fe48fe39700 1 -- 172.21.15.62:0/3477942944 >> 172.21.15.62:0/3477942944 conn(0x7fe4880fb310 msgr2=0x7fe4880fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:09.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.203+0000 7fe48fe39700 1 -- 172.21.15.62:0/3477942944 shutdown_connections 2024-05-08T22:36:09.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.203+0000 7fe48fe39700 1 -- 172.21.15.62:0/3477942944 wait complete. 2024-05-08T22:36:09.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48fe39700 1 Processor -- start 2024-05-08T22:36:09.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48fe39700 1 -- start start 2024-05-08T22:36:09.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48fe39700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4880ffd80 0x7fe488195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48fe39700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488100f70 0x7fe488196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48fe39700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe488101910 0x7fe48819d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48fe39700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48819d840 con 0x7fe4880ffd80 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48fe39700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48819d9b0 con 0x7fe488101910 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48fe39700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48819db70 con 0x7fe488100f70 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48dbd5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4880ffd80 0x7fe488195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48dbd5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4880ffd80 0x7fe488195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43044/0 (socket says 172.21.15.62:43044) 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48d3d4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488100f70 0x7fe488196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48dbd5700 1 -- 172.21.15.62:0/2670246613 learned_addr learned my addr 172.21.15.62:0/2670246613 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.204+0000 7fe48e3d6700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe488101910 0x7fe48819d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe48d3d4700 1 -- 172.21.15.62:0/2670246613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe488101910 msgr2=0x7fe48819d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe48d3d4700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe488101910 0x7fe48819d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe48d3d4700 1 -- 172.21.15.62:0/2670246613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4880ffd80 msgr2=0x7fe488195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe48d3d4700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4880ffd80 0x7fe488195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe48d3d4700 1 -- 172.21.15.62:0/2670246613 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4840095f0 con 0x7fe488100f70 2024-05-08T22:36:09.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe48dbd5700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4880ffd80 0x7fe488195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:09.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe48d3d4700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488100f70 0x7fe488196290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe47800be10 tx=0x7fe47800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe47effd700 1 -- 172.21.15.62:0/2670246613 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478010a90 con 0x7fe488100f70 2024-05-08T22:36:09.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe47effd700 1 -- 172.21.15.62:0/2670246613 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe478014070 con 0x7fe488100f70 2024-05-08T22:36:09.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe48fe39700 1 -- 172.21.15.62:0/2670246613 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4880681c0 con 0x7fe488100f70 2024-05-08T22:36:09.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe47effd700 1 -- 172.21.15.62:0/2670246613 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4780193b0 con 0x7fe488100f70 2024-05-08T22:36:09.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.205+0000 7fe48fe39700 1 -- 172.21.15.62:0/2670246613 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe488068720 con 0x7fe488100f70 2024-05-08T22:36:09.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.206+0000 7fe48fe39700 1 -- 172.21.15.62:0/2670246613 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe488067d10 con 0x7fe488100f70 2024-05-08T22:36:09.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.207+0000 7fe47effd700 1 -- 172.21.15.62:0/2670246613 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe47801a400 con 0x7fe488100f70 2024-05-08T22:36:09.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.208+0000 7fe47effd700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4740a1230 0x7fe4740a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.208+0000 7fe47effd700 1 -- 172.21.15.62:0/2670246613 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe4780cdf90 con 0x7fe488100f70 2024-05-08T22:36:09.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.208+0000 7fe48dbd5700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4740a1230 0x7fe4740a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.208+0000 7fe48dbd5700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4740a1230 0x7fe4740a36f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe484000c00 tx=0x7fe4840046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.212+0000 7fe47effd700 1 -- 172.21.15.62:0/2670246613 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe478054d60 con 0x7fe488100f70 2024-05-08T22:36:09.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.429+0000 7fe48fe39700 1 -- 172.21.15.62:0/2670246613 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe488106c50 con 0x7fe488100f70 2024-05-08T22:36:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.431+0000 7fe47effd700 1 -- 172.21.15.62:0/2670246613 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22481 (secure 0 0 0) 0x7fe47809b170 con 0x7fe488100f70 2024-05-08T22:36:09.431 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:36:09.431 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:36:05.960694+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 -- 172.21.15.62:0/2670246613 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4740a1230 msgr2=0x7fe4740a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4740a1230 0x7fe4740a36f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe484000c00 tx=0x7fe4840046c0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 -- 172.21.15.62:0/2670246613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488100f70 msgr2=0x7fe488196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488100f70 0x7fe488196290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe47800be10 tx=0x7fe47800dc30 comp rx=0 tx=0).stop 2024-05-08T22:36:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 -- 172.21.15.62:0/2670246613 shutdown_connections 2024-05-08T22:36:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fe4740a1230 0x7fe4740a36f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe488101910 0x7fe48819d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4880ffd80 0x7fe488195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 --2- 172.21.15.62:0/2670246613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488100f70 0x7fe488196290 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 -- 172.21.15.62:0/2670246613 >> 172.21.15.62:0/2670246613 conn(0x7fe4880fb310 msgr2=0x7fe4881043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 -- 172.21.15.62:0/2670246613 shutdown_connections 2024-05-08T22:36:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.433+0000 7fe48fe39700 1 -- 172.21.15.62:0/2670246613 wait complete. 2024-05-08T22:36:09.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:31:54.466249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:35:08.296334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:35:09.167901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:36:09.450 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:36:09.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.582+0000 7f64ec52b700 1 -- 172.21.15.62:0/1428570545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4101910 msgr2=0x7f64e41081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.582+0000 7f64ec52b700 1 --2- 172.21.15.62:0/1428570545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4101910 0x7f64e41081c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b0d0 tx=0x7f64dc00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 -- 172.21.15.62:0/1428570545 shutdown_connections 2024-05-08T22:36:09.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 --2- 172.21.15.62:0/1428570545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4101910 0x7f64e41081c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 --2- 172.21.15.62:0/1428570545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64e40ffd80 0x7f64e41001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 --2- 172.21.15.62:0/1428570545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64e4100f70 0x7f64e41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 -- 172.21.15.62:0/1428570545 >> 172.21.15.62:0/1428570545 conn(0x7f64e40fb310 msgr2=0x7f64e40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:09.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 -- 172.21.15.62:0/1428570545 shutdown_connections 2024-05-08T22:36:09.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 -- 172.21.15.62:0/1428570545 wait complete. 2024-05-08T22:36:09.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 Processor -- start 2024-05-08T22:36:09.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 -- start start 2024-05-08T22:36:09.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64e40ffd80 0x7f64e4196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4100f70 0x7f64e4197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64e4101910 0x7f64e419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e419ea80 con 0x7f64e4100f70 2024-05-08T22:36:09.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e419ebc0 con 0x7f64e40ffd80 2024-05-08T22:36:09.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.583+0000 7f64ec52b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e419ed50 con 0x7f64e4101910 2024-05-08T22:36:09.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64e9ac6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4100f70 0x7f64e4197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64e9ac6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4100f70 0x7f64e4197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43068/0 (socket says 172.21.15.62:43068) 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64e9ac6700 1 -- 172.21.15.62:0/3578230128 learned_addr learned my addr 172.21.15.62:0/3578230128 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64eaac8700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64e4101910 0x7f64e419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64ea2c7700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64e40ffd80 0x7f64e4196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64e9ac6700 1 -- 172.21.15.62:0/3578230128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64e4101910 msgr2=0x7f64e419e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64e9ac6700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64e4101910 0x7f64e419e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64e9ac6700 1 -- 172.21.15.62:0/3578230128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64e40ffd80 msgr2=0x7f64e4196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64e9ac6700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64e40ffd80 0x7f64e4196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64e9ac6700 1 -- 172.21.15.62:0/3578230128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc009d00 con 0x7f64e4100f70 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64e9ac6700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4100f70 0x7f64e4197490 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f64d400c920 tx=0x7f64d400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64db7fe700 1 -- 172.21.15.62:0/3578230128 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4007b00 con 0x7f64e4100f70 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64db7fe700 1 -- 172.21.15.62:0/3578230128 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64d4004cf0 con 0x7f64e4100f70 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64ec52b700 1 -- 172.21.15.62:0/3578230128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e419efd0 con 0x7f64e4100f70 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64db7fe700 1 -- 172.21.15.62:0/3578230128 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4005520 con 0x7f64e4100f70 2024-05-08T22:36:09.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.584+0000 7f64ec52b700 1 -- 172.21.15.62:0/3578230128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e419f4b0 con 0x7f64e4100f70 2024-05-08T22:36:09.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.585+0000 7f64ec52b700 1 -- 172.21.15.62:0/3578230128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e4068160 con 0x7f64e4100f70 2024-05-08T22:36:09.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.586+0000 7f64db7fe700 1 -- 172.21.15.62:0/3578230128 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f64d4004750 con 0x7f64e4100f70 2024-05-08T22:36:09.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.587+0000 7f64db7fe700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64d00a11b0 0x7f64d00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.587+0000 7f64ea2c7700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64d00a11b0 0x7f64d00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.587+0000 7f64db7fe700 1 -- 172.21.15.62:0/3578230128 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f64d40cc760 con 0x7f64e4100f70 2024-05-08T22:36:09.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.587+0000 7f64ea2c7700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64d00a11b0 0x7f64d00a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005fd0 tx=0x7f64e0005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.591+0000 7f64db7fe700 1 -- 172.21.15.62:0/3578230128 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f64d4095d40 con 0x7f64e4100f70 2024-05-08T22:36:09.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.797+0000 7f64ec52b700 1 -- 172.21.15.62:0/3578230128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64e419f700 con 0x7f64e4100f70 2024-05-08T22:36:09.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.798+0000 7f64db7fe700 1 -- 172.21.15.62:0/3578230128 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64d401d0a0 con 0x7f64e4100f70 2024-05-08T22:36:09.798 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-05-08T22:36:09.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 -- 172.21.15.62:0/3578230128 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64d00a11b0 msgr2=0x7f64d00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64d00a11b0 0x7f64d00a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005fd0 tx=0x7f64e0005dd0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 -- 172.21.15.62:0/3578230128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4100f70 msgr2=0x7f64e4197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4100f70 0x7f64e4197490 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f64d400c920 tx=0x7f64d400cce0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 -- 172.21.15.62:0/3578230128 shutdown_connections 2024-05-08T22:36:09.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64e4101910 0x7f64e419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f64d00a11b0 0x7f64d00a3670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64e40ffd80 0x7f64e4196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 --2- 172.21.15.62:0/3578230128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4100f70 0x7f64e4197490 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 -- 172.21.15.62:0/3578230128 >> 172.21.15.62:0/3578230128 conn(0x7f64e40fb310 msgr2=0x7f64e40fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:09.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 -- 172.21.15.62:0/3578230128 shutdown_connections 2024-05-08T22:36:09.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.800+0000 7f64ec52b700 1 -- 172.21.15.62:0/3578230128 wait complete. 2024-05-08T22:36:09.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: pgmap v253: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 7.3 KiB/s, 0 objects/s recovering 2024-05-08T22:36:09.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3072710034' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T22:36:09.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:36:09.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:36:09.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:09 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2670246613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:36:09.813 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:36:09.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: pgmap v253: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 7.3 KiB/s, 0 objects/s recovering 2024-05-08T22:36:09.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3072710034' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T22:36:09.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:36:09.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:36:09.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:09 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2670246613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:36:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: pgmap v253: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 7.3 KiB/s, 0 objects/s recovering 2024-05-08T22:36:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3072710034' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T22:36:09.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:36:09.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:36:09.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:36:09.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:09 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2670246613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:36:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.942+0000 7feeeee7c700 1 -- 172.21.15.62:0/2331280528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8101fc0 msgr2=0x7feee8102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.942+0000 7feeeee7c700 1 --2- 172.21.15.62:0/2331280528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8101fc0 0x7feee8102440 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7feed4009990 tx=0x7feed4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.942+0000 7feeeee7c700 1 -- 172.21.15.62:0/2331280528 shutdown_connections 2024-05-08T22:36:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.942+0000 7feeeee7c700 1 --2- 172.21.15.62:0/2331280528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee8102980 0x7feee81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.942+0000 7feeeee7c700 1 --2- 172.21.15.62:0/2331280528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8100e70 0x7feee8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.942+0000 7feeeee7c700 1 --2- 172.21.15.62:0/2331280528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8101fc0 0x7feee8102440 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.942+0000 7feeeee7c700 1 -- 172.21.15.62:0/2331280528 >> 172.21.15.62:0/2331280528 conn(0x7feee80fc400 msgr2=0x7feee80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.942+0000 7feeeee7c700 1 -- 172.21.15.62:0/2331280528 shutdown_connections 2024-05-08T22:36:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.942+0000 7feeeee7c700 1 -- 172.21.15.62:0/2331280528 wait complete. 2024-05-08T22:36:09.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.943+0000 7feeeee7c700 1 Processor -- start 2024-05-08T22:36:09.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.943+0000 7feeeee7c700 1 -- start start 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.943+0000 7feeeee7c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee8100e70 0x7feee8074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.943+0000 7feeeee7c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8101fc0 0x7feee8073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.943+0000 7feeeee7c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8102980 0x7feee80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.943+0000 7feeeee7c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee8074070 con 0x7feee8101fc0 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.943+0000 7feeeee7c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80741e0 con 0x7feee8100e70 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.943+0000 7feeeee7c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee81a0e90 con 0x7feee8102980 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.943+0000 7feee7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8101fc0 0x7feee8073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.943+0000 7feee7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8101fc0 0x7feee8073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43088/0 (socket says 172.21.15.62:43088) 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feee7fff700 1 -- 172.21.15.62:0/4241788173 learned_addr learned my addr 172.21.15.62:0/4241788173 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feeecc18700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee8100e70 0x7feee8074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feeed419700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8102980 0x7feee80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feee7fff700 1 -- 172.21.15.62:0/4241788173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8102980 msgr2=0x7feee80738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feee7fff700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8102980 0x7feee80738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feee7fff700 1 -- 172.21.15.62:0/4241788173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee8100e70 msgr2=0x7feee8074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feee7fff700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee8100e70 0x7feee8074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feee7fff700 1 -- 172.21.15.62:0/4241788173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed40095f0 con 0x7feee8101fc0 2024-05-08T22:36:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feee7fff700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8101fc0 0x7feee8073390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7feed4005910 tx=0x7feed4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feee5ffb700 1 -- 172.21.15.62:0/4241788173 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed401d070 con 0x7feee8101fc0 2024-05-08T22:36:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feee5ffb700 1 -- 172.21.15.62:0/4241788173 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feed4003f60 con 0x7feee8101fc0 2024-05-08T22:36:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feee5ffb700 1 -- 172.21.15.62:0/4241788173 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4022c70 con 0x7feee8101fc0 2024-05-08T22:36:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feeeee7c700 1 -- 172.21.15.62:0/4241788173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee81a10c0 con 0x7feee8101fc0 2024-05-08T22:36:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.944+0000 7feeeee7c700 1 -- 172.21.15.62:0/4241788173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee81a1430 con 0x7feee8101fc0 2024-05-08T22:36:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.945+0000 7feeeee7c700 1 -- 172.21.15.62:0/4241788173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee81082e0 con 0x7feee8101fc0 2024-05-08T22:36:09.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.946+0000 7feee5ffb700 1 -- 172.21.15.62:0/4241788173 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7feed4022470 con 0x7feee8101fc0 2024-05-08T22:36:09.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.947+0000 7feee5ffb700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7feed80a11b0 0x7feed80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:09.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.947+0000 7feee5ffb700 1 -- 172.21.15.62:0/4241788173 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feed40ce650 con 0x7feee8101fc0 2024-05-08T22:36:09.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.947+0000 7feeecc18700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7feed80a11b0 0x7feed80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:09.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.947+0000 7feeecc18700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7feed80a11b0 0x7feed80a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feedc005950 tx=0x7feedc00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:09.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:09.950+0000 7feee5ffb700 1 -- 172.21.15.62:0/4241788173 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feed4097cc0 con 0x7feee8101fc0 2024-05-08T22:36:10.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.158+0000 7feeeee7c700 1 -- 172.21.15.62:0/4241788173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feee810c7d0 con 0x7feee8101fc0 2024-05-08T22:36:10.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.158+0000 7feee5ffb700 1 -- 172.21.15.62:0/4241788173 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7feed409b2e0 con 0x7feee8101fc0 2024-05-08T22:36:10.158 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-05-08T22:36:10.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 -- 172.21.15.62:0/4241788173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7feed80a11b0 msgr2=0x7feed80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7feed80a11b0 0x7feed80a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feedc005950 tx=0x7feedc00a300 comp rx=0 tx=0).stop 2024-05-08T22:36:10.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 -- 172.21.15.62:0/4241788173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8101fc0 msgr2=0x7feee8073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8101fc0 0x7feee8073390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7feed4005910 tx=0x7feed4004730 comp rx=0 tx=0).stop 2024-05-08T22:36:10.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 -- 172.21.15.62:0/4241788173 shutdown_connections 2024-05-08T22:36:10.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8102980 0x7feee80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7feed80a11b0 0x7feed80a3670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feee8100e70 0x7feee8074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 --2- 172.21.15.62:0/4241788173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8101fc0 0x7feee8073390 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 -- 172.21.15.62:0/4241788173 >> 172.21.15.62:0/4241788173 conn(0x7feee80fc400 msgr2=0x7feee8105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:10.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 -- 172.21.15.62:0/4241788173 shutdown_connections 2024-05-08T22:36:10.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.160+0000 7feeeee7c700 1 -- 172.21.15.62:0/4241788173 wait complete. 2024-05-08T22:36:10.173 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:36:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.302+0000 7f4a5909f700 1 -- 172.21.15.62:0/1364367149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54101930 msgr2=0x7f4a541081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.302+0000 7f4a5909f700 1 --2- 172.21.15.62:0/1364367149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54101930 0x7f4a541081e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00b0d0 tx=0x7f4a4c00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.302+0000 7f4a5909f700 1 -- 172.21.15.62:0/1364367149 shutdown_connections 2024-05-08T22:36:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.302+0000 7f4a5909f700 1 --2- 172.21.15.62:0/1364367149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a54101930 0x7f4a541081e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.302+0000 7f4a5909f700 1 --2- 172.21.15.62:0/1364367149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a540ffda0 0x7f4a541001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.302+0000 7f4a5909f700 1 --2- 172.21.15.62:0/1364367149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a54100f90 0x7f4a541013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.302+0000 7f4a5909f700 1 -- 172.21.15.62:0/1364367149 >> 172.21.15.62:0/1364367149 conn(0x7f4a540fb330 msgr2=0x7f4a540fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.302+0000 7f4a5909f700 1 -- 172.21.15.62:0/1364367149 shutdown_connections 2024-05-08T22:36:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.302+0000 7f4a5909f700 1 -- 172.21.15.62:0/1364367149 wait complete. 2024-05-08T22:36:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.302+0000 7f4a5909f700 1 Processor -- start 2024-05-08T22:36:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a5909f700 1 -- start start 2024-05-08T22:36:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a5909f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a540ffda0 0x7f4a54069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a5909f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a54100f90 0x7f4a5419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a5909f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a54101930 0x7f4a5419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a5909f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419ea30 con 0x7f4a540ffda0 2024-05-08T22:36:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a5909f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419eba0 con 0x7f4a54100f90 2024-05-08T22:36:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a5909f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419ed60 con 0x7f4a54101930 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a52d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a540ffda0 0x7f4a54069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a52d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a540ffda0 0x7f4a54069f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43096/0 (socket says 172.21.15.62:43096) 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a52d9d700 1 -- 172.21.15.62:0/1271404888 learned_addr learned my addr 172.21.15.62:0/1271404888 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a5259c700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a54100f90 0x7f4a5419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a5359e700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a54101930 0x7f4a5419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a52d9d700 1 -- 172.21.15.62:0/1271404888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a54101930 msgr2=0x7f4a5419e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a52d9d700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a54101930 0x7f4a5419e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a52d9d700 1 -- 172.21.15.62:0/1271404888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a54100f90 msgr2=0x7f4a5419bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a52d9d700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a54100f90 0x7f4a5419bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.303+0000 7f4a52d9d700 1 -- 172.21.15.62:0/1271404888 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c009d00 con 0x7f4a540ffda0 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.304+0000 7f4a52d9d700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a540ffda0 0x7f4a54069f90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800b500 tx=0x7f4a4800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.304+0000 7f4a3bfff700 1 -- 172.21.15.62:0/1271404888 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a48011830 con 0x7f4a540ffda0 2024-05-08T22:36:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.304+0000 7f4a3bfff700 1 -- 172.21.15.62:0/1271404888 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a48011e50 con 0x7f4a540ffda0 2024-05-08T22:36:10.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.304+0000 7f4a5909f700 1 -- 172.21.15.62:0/1271404888 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5419efe0 con 0x7f4a540ffda0 2024-05-08T22:36:10.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.304+0000 7f4a3bfff700 1 -- 172.21.15.62:0/1271404888 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800f2e0 con 0x7f4a540ffda0 2024-05-08T22:36:10.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.304+0000 7f4a5909f700 1 -- 172.21.15.62:0/1271404888 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5419f4e0 con 0x7f4a540ffda0 2024-05-08T22:36:10.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.305+0000 7f4a5909f700 1 -- 172.21.15.62:0/1271404888 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a54107150 con 0x7f4a540ffda0 2024-05-08T22:36:10.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.306+0000 7f4a3bfff700 1 -- 172.21.15.62:0/1271404888 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4a48010580 con 0x7f4a540ffda0 2024-05-08T22:36:10.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.306+0000 7f4a3bfff700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4a3c0a11b0 0x7f4a3c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.306+0000 7f4a3bfff700 1 -- 172.21.15.62:0/1271404888 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4a480cc3c0 con 0x7f4a540ffda0 2024-05-08T22:36:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.306+0000 7f4a5259c700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4a3c0a11b0 0x7f4a3c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.307+0000 7f4a5259c700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4a3c0a11b0 0x7f4a3c0a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a5406a550 tx=0x7f4a4400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.311+0000 7f4a3bfff700 1 -- 172.21.15.62:0/1271404888 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4a48095b10 con 0x7f4a540ffda0 2024-05-08T22:36:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.519+0000 7f4a5909f700 1 -- 172.21.15.62:0/1271404888 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a5410b7e0 con 0x7f4a540ffda0 2024-05-08T22:36:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.519+0000 7f4a3bfff700 1 -- 172.21.15.62:0/1271404888 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a48099130 con 0x7f4a540ffda0 2024-05-08T22:36:10.519 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-05-08T22:36:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.521+0000 7f4a5909f700 1 -- 172.21.15.62:0/1271404888 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4a3c0a11b0 msgr2=0x7f4a3c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.521+0000 7f4a5909f700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4a3c0a11b0 0x7f4a3c0a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a5406a550 tx=0x7f4a4400b410 comp rx=0 tx=0).stop 2024-05-08T22:36:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.522+0000 7f4a5909f700 1 -- 172.21.15.62:0/1271404888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a540ffda0 msgr2=0x7f4a54069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.522+0000 7f4a5909f700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a540ffda0 0x7f4a54069f90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800b500 tx=0x7f4a4800b810 comp rx=0 tx=0).stop 2024-05-08T22:36:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.522+0000 7f4a5909f700 1 -- 172.21.15.62:0/1271404888 shutdown_connections 2024-05-08T22:36:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.522+0000 7f4a5909f700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a54101930 0x7f4a5419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.522+0000 7f4a5909f700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a540ffda0 0x7f4a54069f90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.522+0000 7f4a5909f700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f4a3c0a11b0 0x7f4a3c0a3670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.522+0000 7f4a5909f700 1 --2- 172.21.15.62:0/1271404888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a54100f90 0x7f4a5419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.522+0000 7f4a5909f700 1 -- 172.21.15.62:0/1271404888 >> 172.21.15.62:0/1271404888 conn(0x7f4a540fb330 msgr2=0x7f4a54104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.522+0000 7f4a5909f700 1 -- 172.21.15.62:0/1271404888 shutdown_connections 2024-05-08T22:36:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.522+0000 7f4a5909f700 1 -- 172.21.15.62:0/1271404888 wait complete. 2024-05-08T22:36:10.534 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:36:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.660+0000 7f8704414700 1 -- 172.21.15.62:0/467546261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86fc100f70 msgr2=0x7f86fc1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.660+0000 7f8704414700 1 --2- 172.21.15.62:0/467546261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86fc100f70 0x7f86fc1013d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0099c0 tx=0x7f86ec009cd0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.660+0000 7f8704414700 1 -- 172.21.15.62:0/467546261 shutdown_connections 2024-05-08T22:36:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.660+0000 7f8704414700 1 --2- 172.21.15.62:0/467546261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86fc101910 0x7f86fc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.660+0000 7f8704414700 1 --2- 172.21.15.62:0/467546261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86fc0ffd80 0x7f86fc1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.660+0000 7f8704414700 1 --2- 172.21.15.62:0/467546261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86fc100f70 0x7f86fc1013d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.660+0000 7f8704414700 1 -- 172.21.15.62:0/467546261 >> 172.21.15.62:0/467546261 conn(0x7f86fc0fb310 msgr2=0x7f86fc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.660+0000 7f8704414700 1 -- 172.21.15.62:0/467546261 shutdown_connections 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.660+0000 7f8704414700 1 -- 172.21.15.62:0/467546261 wait complete. 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f8704414700 1 Processor -- start 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f8704414700 1 -- start start 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f8704414700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86fc0ffd80 0x7f86fc10a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f8704414700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86fc100f70 0x7f86fc108cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f8704414700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86fc101910 0x7f86fc109210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f8704414700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc1099b0 con 0x7f86fc101910 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f8704414700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc109b20 con 0x7f86fc100f70 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f8704414700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc1a0e40 con 0x7f86fc0ffd80 2024-05-08T22:36:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f87029b1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86fc101910 0x7f86fc109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f87029b1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86fc101910 0x7f86fc109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43116/0 (socket says 172.21.15.62:43116) 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f87029b1700 1 -- 172.21.15.62:0/2971129957 learned_addr learned my addr 172.21.15.62:0/2971129957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f87021b0700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86fc0ffd80 0x7f86fc10a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.661+0000 7f87019af700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86fc100f70 0x7f86fc108cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f87029b1700 1 -- 172.21.15.62:0/2971129957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86fc0ffd80 msgr2=0x7f86fc10a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f87029b1700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86fc0ffd80 0x7f86fc10a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f87029b1700 1 -- 172.21.15.62:0/2971129957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86fc100f70 msgr2=0x7f86fc108cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f87029b1700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86fc100f70 0x7f86fc108cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f87029b1700 1 -- 172.21.15.62:0/2971129957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0095f0 con 0x7f86fc101910 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f87019af700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86fc100f70 0x7f86fc108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f87029b1700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86fc101910 0x7f86fc109210 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f86f4007ee0 tx=0x7f86f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f86f37fe700 1 -- 172.21.15.62:0/2971129957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4017070 con 0x7f86fc101910 2024-05-08T22:36:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f86f37fe700 1 -- 172.21.15.62:0/2971129957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86f400f040 con 0x7f86fc101910 2024-05-08T22:36:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f8704414700 1 -- 172.21.15.62:0/2971129957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86fc1a10d0 con 0x7f86fc101910 2024-05-08T22:36:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f86f37fe700 1 -- 172.21.15.62:0/2971129957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4005740 con 0x7f86fc101910 2024-05-08T22:36:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.662+0000 7f8704414700 1 -- 172.21.15.62:0/2971129957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86fc1a1590 con 0x7f86fc101910 2024-05-08T22:36:10.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.663+0000 7f8704414700 1 -- 172.21.15.62:0/2971129957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86fc1071d0 con 0x7f86fc101910 2024-05-08T22:36:10.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.664+0000 7f86f37fe700 1 -- 172.21.15.62:0/2971129957 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f86f4004c40 con 0x7f86fc101910 2024-05-08T22:36:10.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.665+0000 7f86f37fe700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86e80a11b0 0x7f86e80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:10.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.665+0000 7f87021b0700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86e80a11b0 0x7f86e80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:10.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.665+0000 7f86f37fe700 1 -- 172.21.15.62:0/2971129957 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86f40cc860 con 0x7f86fc101910 2024-05-08T22:36:10.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.665+0000 7f87021b0700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86e80a11b0 0x7f86e80a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f86fc100dd0 tx=0x7f86f8007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:10.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.668+0000 7f86f37fe700 1 -- 172.21.15.62:0/2971129957 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f86f4095ea0 con 0x7f86fc101910 2024-05-08T22:36:10.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:10 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3578230128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:36:10.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:10 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/4241788173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:36:10.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:10 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3578230128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:36:10.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:10 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/4241788173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:36:10.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:10 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3578230128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:36:10.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:10 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/4241788173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.922+0000 7f8704414700 1 -- 172.21.15.62:0/2971129957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86fc067d10 con 0x7f86fc101910 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.922+0000 7f86f37fe700 1 -- 172.21.15.62:0/2971129957 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10897 (secure 0 0 0) 0x7f86f40994c0 con 0x7f86fc101910 2024-05-08T22:36:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:36:10.923 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:35:09.803459+0000","modified":"2024-05-08T22:35:16.261557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24529,"mds_2":24505,"mds_3":24503,"mds_4":14661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/1756743060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1756743060},{"type":"v1","addr":"172.21.15.62:6839","nonce":1756743060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14661":{"gid":14661,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/780017086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":780017086},{"type":"v1","addr":"172.21.15.62:6841","nonce":780017086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.79:6839/3534785027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":3534785027},{"type":"v1","addr":"172.21.15.79:6839","nonce":3534785027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/3474672912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":3474672912},{"type":"v1","addr":"172.21.15.66:6839","nonce":3474672912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:36:10.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.924+0000 7f8704414700 1 -- 172.21.15.62:0/2971129957 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86e80a11b0 msgr2=0x7f86e80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.924+0000 7f8704414700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86e80a11b0 0x7f86e80a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f86fc100dd0 tx=0x7f86f8007400 comp rx=0 tx=0).stop 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.924+0000 7f8704414700 1 -- 172.21.15.62:0/2971129957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86fc101910 msgr2=0x7f86fc109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.924+0000 7f8704414700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86fc101910 0x7f86fc109210 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f86f4007ee0 tx=0x7f86f400d3b0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.924+0000 7f8704414700 1 -- 172.21.15.62:0/2971129957 shutdown_connections 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.924+0000 7f8704414700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86fc101910 0x7f86fc109210 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.924+0000 7f8704414700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86fc0ffd80 0x7f86fc10a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.924+0000 7f8704414700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f86e80a11b0 0x7f86e80a3670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.924+0000 7f8704414700 1 --2- 172.21.15.62:0/2971129957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86fc100f70 0x7f86fc108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.924+0000 7f8704414700 1 -- 172.21.15.62:0/2971129957 >> 172.21.15.62:0/2971129957 conn(0x7f86fc0fb310 msgr2=0x7f86fc104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.924+0000 7f8704414700 1 -- 172.21.15.62:0/2971129957 shutdown_connections 2024-05-08T22:36:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:10.925+0000 7f8704414700 1 -- 172.21.15.62:0/2971129957 wait complete. 2024-05-08T22:36:10.926 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-05-08T22:36:10.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:11.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.064+0000 7fbdda056700 1 -- 172.21.15.62:0/1032249263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100e80 msgr2=0x7fbdd41012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.064+0000 7fbdda056700 1 --2- 172.21.15.62:0/1032249263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100e80 0x7fbdd41012a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009990 tx=0x7fbdc4009ca0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.064+0000 7fbdda056700 1 -- 172.21.15.62:0/1032249263 shutdown_connections 2024-05-08T22:36:11.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.064+0000 7fbdda056700 1 --2- 172.21.15.62:0/1032249263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdd4102990 0x7fbdd41092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.064+0000 7fbdda056700 1 --2- 172.21.15.62:0/1032249263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100e80 0x7fbdd41012a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.064+0000 7fbdda056700 1 --2- 172.21.15.62:0/1032249263 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdd4101fd0 0x7fbdd4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.064+0000 7fbdda056700 1 -- 172.21.15.62:0/1032249263 >> 172.21.15.62:0/1032249263 conn(0x7fbdd40fc410 msgr2=0x7fbdd40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:11.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.064+0000 7fbdda056700 1 -- 172.21.15.62:0/1032249263 shutdown_connections 2024-05-08T22:36:11.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.064+0000 7fbdda056700 1 -- 172.21.15.62:0/1032249263 wait complete. 2024-05-08T22:36:11.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.065+0000 7fbdda056700 1 Processor -- start 2024-05-08T22:36:11.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.065+0000 7fbdda056700 1 -- start start 2024-05-08T22:36:11.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdda056700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100e80 0x7fbdd4196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdda056700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdd4101fd0 0x7fbdd4197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdda056700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdd4102990 0x7fbdd419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdda056700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd419e8f0 con 0x7fbdd4100e80 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdda056700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd419ea30 con 0x7fbdd4101fd0 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdda056700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd419ebc0 con 0x7fbdd4102990 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100e80 0x7fbdd4196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100e80 0x7fbdd4196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43148/0 (socket says 172.21.15.62:43148) 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd37fe700 1 -- 172.21.15.62:0/2669931244 learned_addr learned my addr 172.21.15.62:0/2669931244 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd3fff700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdd4102990 0x7fbdd419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd2ffd700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdd4101fd0 0x7fbdd4197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd37fe700 1 -- 172.21.15.62:0/2669931244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdd4102990 msgr2=0x7fbdd419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd37fe700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdd4102990 0x7fbdd419e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd37fe700 1 -- 172.21.15.62:0/2669931244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdd4101fd0 msgr2=0x7fbdd4197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd37fe700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdd4101fd0 0x7fbdd4197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd37fe700 1 -- 172.21.15.62:0/2669931244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc40095f0 con 0x7fbdd4100e80 2024-05-08T22:36:11.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd37fe700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100e80 0x7fbdd4196dc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400b5c0 tx=0x7fbdc4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd0ff9700 1 -- 172.21.15.62:0/2669931244 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc401d070 con 0x7fbdd4100e80 2024-05-08T22:36:11.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdd0ff9700 1 -- 172.21.15.62:0/2669931244 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdc4004060 con 0x7fbdd4100e80 2024-05-08T22:36:11.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.066+0000 7fbdda056700 1 -- 172.21.15.62:0/2669931244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd419ee40 con 0x7fbdd4100e80 2024-05-08T22:36:11.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.067+0000 7fbdd0ff9700 1 -- 172.21.15.62:0/2669931244 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4022e10 con 0x7fbdd4100e80 2024-05-08T22:36:11.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.067+0000 7fbdda056700 1 -- 172.21.15.62:0/2669931244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd419f1b0 con 0x7fbdd4100e80 2024-05-08T22:36:11.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.068+0000 7fbdda056700 1 -- 172.21.15.62:0/2669931244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd40681d0 con 0x7fbdd4100e80 2024-05-08T22:36:11.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.069+0000 7fbdd0ff9700 1 -- 172.21.15.62:0/2669931244 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbdc4022570 con 0x7fbdd4100e80 2024-05-08T22:36:11.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.069+0000 7fbdd0ff9700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbdbc0a11b0 0x7fbdbc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.070+0000 7fbdd2ffd700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbdbc0a11b0 0x7fbdbc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.070+0000 7fbdd0ff9700 1 -- 172.21.15.62:0/2669931244 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbdc40cddf0 con 0x7fbdd4100e80 2024-05-08T22:36:11.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.070+0000 7fbdd2ffd700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbdbc0a11b0 0x7fbdbc0a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8005950 tx=0x7fbdc80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.074+0000 7fbdd0ff9700 1 -- 172.21.15.62:0/2669931244 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbdd40681d0 con 0x7fbdd4100e80 2024-05-08T22:36:11.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.283+0000 7fbdda056700 1 -- 172.21.15.62:0/2669931244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdd40681d0 con 0x7fbdd4100e80 2024-05-08T22:36:11.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.285+0000 7fbdd0ff9700 1 -- 172.21.15.62:0/2669931244 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22481 (secure 0 0 0) 0x7fbdd40681d0 con 0x7fbdd4100e80 2024-05-08T22:36:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:36:11.285 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:36:05.960694+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:11.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 -- 172.21.15.62:0/2669931244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbdbc0a11b0 msgr2=0x7fbdbc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbdbc0a11b0 0x7fbdbc0a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8005950 tx=0x7fbdc80058e0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 -- 172.21.15.62:0/2669931244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100e80 msgr2=0x7fbdd4196dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100e80 0x7fbdd4196dc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400b5c0 tx=0x7fbdc4005e80 comp rx=0 tx=0).stop 2024-05-08T22:36:11.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 -- 172.21.15.62:0/2669931244 shutdown_connections 2024-05-08T22:36:11.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdd4102990 0x7fbdd419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100e80 0x7fbdd4196dc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbdbc0a11b0 0x7fbdbc0a3670 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 --2- 172.21.15.62:0/2669931244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbdd4101fd0 0x7fbdd4197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 -- 172.21.15.62:0/2669931244 >> 172.21.15.62:0/2669931244 conn(0x7fbdd40fc410 msgr2=0x7fbdd41053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:11.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 -- 172.21.15.62:0/2669931244 shutdown_connections 2024-05-08T22:36:11.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.287+0000 7fbdda056700 1 -- 172.21.15.62:0/2669931244 wait complete. 2024-05-08T22:36:11.300 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:31:54.466249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:35:08.296334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:35:09.167901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:36:11.300 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:36:11.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.430+0000 7fbec073c700 1 -- 172.21.15.62:0/522756065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8100f80 msgr2=0x7fbeb81013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.430+0000 7fbec073c700 1 --2- 172.21.15.62:0/522756065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8100f80 0x7fbeb81013e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40099c0 tx=0x7fbeb4009cd0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.430+0000 7fbec073c700 1 -- 172.21.15.62:0/522756065 shutdown_connections 2024-05-08T22:36:11.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.430+0000 7fbec073c700 1 --2- 172.21.15.62:0/522756065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeb8101920 0x7fbeb81081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.430+0000 7fbec073c700 1 --2- 172.21.15.62:0/522756065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeb80ffd90 0x7fbeb81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.430+0000 7fbec073c700 1 --2- 172.21.15.62:0/522756065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8100f80 0x7fbeb81013e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.430+0000 7fbec073c700 1 -- 172.21.15.62:0/522756065 >> 172.21.15.62:0/522756065 conn(0x7fbeb80fb300 msgr2=0x7fbeb80fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:11.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.430+0000 7fbec073c700 1 -- 172.21.15.62:0/522756065 shutdown_connections 2024-05-08T22:36:11.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.430+0000 7fbec073c700 1 -- 172.21.15.62:0/522756065 wait complete. 2024-05-08T22:36:11.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.430+0000 7fbec073c700 1 Processor -- start 2024-05-08T22:36:11.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.430+0000 7fbec073c700 1 -- start start 2024-05-08T22:36:11.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbec073c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeb80ffd90 0x7fbeb8103c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbec073c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeb8100f80 0x7fbeb8102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbec073c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 0x7fbeb81028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbec073c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8102fa0 con 0x7fbeb8101920 2024-05-08T22:36:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbec073c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8103110 con 0x7fbeb8100f80 2024-05-08T22:36:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbec073c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb81a0eb0 con 0x7fbeb80ffd90 2024-05-08T22:36:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebecd9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 0x7fbeb81028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebecd9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 0x7fbeb81028a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43160/0 (socket says 172.21.15.62:43160) 2024-05-08T22:36:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebecd9700 1 -- 172.21.15.62:0/216602812 learned_addr learned my addr 172.21.15.62:0/216602812 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebdcd7700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeb8100f80 0x7fbeb8102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebe4d8700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeb80ffd90 0x7fbeb8103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebecd9700 1 -- 172.21.15.62:0/216602812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeb80ffd90 msgr2=0x7fbeb8103c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebecd9700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeb80ffd90 0x7fbeb8103c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebecd9700 1 -- 172.21.15.62:0/216602812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeb8100f80 msgr2=0x7fbeb8102360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebecd9700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeb8100f80 0x7fbeb8102360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebecd9700 1 -- 172.21.15.62:0/216602812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb40095f0 con 0x7fbeb8101920 2024-05-08T22:36:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebdcd7700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeb8100f80 0x7fbeb8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbebecd9700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 0x7fbeb81028a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000ef30 tx=0x7fbeb000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.431+0000 7fbeaf7fe700 1 -- 172.21.15.62:0/216602812 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0018070 con 0x7fbeb8101920 2024-05-08T22:36:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.432+0000 7fbeaf7fe700 1 -- 172.21.15.62:0/216602812 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb000f040 con 0x7fbeb8101920 2024-05-08T22:36:11.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.432+0000 7fbec073c700 1 -- 172.21.15.62:0/216602812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb81a10b0 con 0x7fbeb8101920 2024-05-08T22:36:11.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.432+0000 7fbeaf7fe700 1 -- 172.21.15.62:0/216602812 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb000cd00 con 0x7fbeb8101920 2024-05-08T22:36:11.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.432+0000 7fbec073c700 1 -- 172.21.15.62:0/216602812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb81a15c0 con 0x7fbeb8101920 2024-05-08T22:36:11.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.432+0000 7fbec073c700 1 -- 172.21.15.62:0/216602812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb8067480 con 0x7fbeb8101920 2024-05-08T22:36:11.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.434+0000 7fbeaf7fe700 1 -- 172.21.15.62:0/216602812 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbeb0003a00 con 0x7fbeb8101920 2024-05-08T22:36:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.434+0000 7fbeaf7fe700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbea40a11b0 0x7fbea40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.434+0000 7fbeaf7fe700 1 -- 172.21.15.62:0/216602812 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbeb00ccf20 con 0x7fbeb8101920 2024-05-08T22:36:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.434+0000 7fbebe4d8700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbea40a11b0 0x7fbea40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.435+0000 7fbebe4d8700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbea40a11b0 0x7fbea40a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8100de0 tx=0x7fbea8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.438+0000 7fbeaf7fe700 1 -- 172.21.15.62:0/216602812 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbeb00965f0 con 0x7fbeb8101920 2024-05-08T22:36:11.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.644+0000 7fbec073c700 1 -- 172.21.15.62:0/216602812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbeb810b780 con 0x7fbeb8101920 2024-05-08T22:36:11.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.645+0000 7fbeaf7fe700 1 -- 172.21.15.62:0/216602812 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbeb0099c10 con 0x7fbeb8101920 2024-05-08T22:36:11.645 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-05-08T22:36:11.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 -- 172.21.15.62:0/216602812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbea40a11b0 msgr2=0x7fbea40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbea40a11b0 0x7fbea40a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8100de0 tx=0x7fbea8009260 comp rx=0 tx=0).stop 2024-05-08T22:36:11.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 -- 172.21.15.62:0/216602812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 msgr2=0x7fbeb81028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 0x7fbeb81028a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000ef30 tx=0x7fbeb000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 -- 172.21.15.62:0/216602812 shutdown_connections 2024-05-08T22:36:11.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 0x7fbeb81028a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeb80ffd90 0x7fbeb8103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fbea40a11b0 0x7fbea40a3670 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 --2- 172.21.15.62:0/216602812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbeb8100f80 0x7fbeb8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 -- 172.21.15.62:0/216602812 >> 172.21.15.62:0/216602812 conn(0x7fbeb80fb300 msgr2=0x7fbeb80fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:11.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 -- 172.21.15.62:0/216602812 shutdown_connections 2024-05-08T22:36:11.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.647+0000 7fbec073c700 1 -- 172.21.15.62:0/216602812 wait complete. 2024-05-08T22:36:11.658 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:11 smithi062 ceph-mon[42571]: pgmap v254: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 7.3 KiB/s, 1 keys/s, 2 objects/s recovering 2024-05-08T22:36:11.658 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:11 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1271404888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:36:11.658 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:11 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2971129957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:36:11.658 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:11 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2669931244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:36:11.659 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:36:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.789+0000 7fc73690c700 1 -- 172.21.15.62:0/2939491405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7300ffd60 msgr2=0x7fc730100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.789+0000 7fc73690c700 1 --2- 172.21.15.62:0/2939491405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7300ffd60 0x7fc730100180 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc7200099c0 tx=0x7fc720009cd0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.789+0000 7fc73690c700 1 -- 172.21.15.62:0/2939491405 shutdown_connections 2024-05-08T22:36:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.789+0000 7fc73690c700 1 --2- 172.21.15.62:0/2939491405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7301018f0 0x7fc7301081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.789+0000 7fc73690c700 1 --2- 172.21.15.62:0/2939491405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7300ffd60 0x7fc730100180 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.789+0000 7fc73690c700 1 --2- 172.21.15.62:0/2939491405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc730100f50 0x7fc7301013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.789+0000 7fc73690c700 1 -- 172.21.15.62:0/2939491405 >> 172.21.15.62:0/2939491405 conn(0x7fc7300fb290 msgr2=0x7fc7300fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.789+0000 7fc73690c700 1 -- 172.21.15.62:0/2939491405 shutdown_connections 2024-05-08T22:36:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.789+0000 7fc73690c700 1 -- 172.21.15.62:0/2939491405 wait complete. 2024-05-08T22:36:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.789+0000 7fc73690c700 1 Processor -- start 2024-05-08T22:36:11.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.789+0000 7fc73690c700 1 -- start start 2024-05-08T22:36:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc73690c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7300ffd60 0x7fc730068c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc73690c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc730100f50 0x7fc73019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc73690c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7301018f0 0x7fc73019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc73690c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73019e900 con 0x7fc7300ffd60 2024-05-08T22:36:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc73690c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73019ea70 con 0x7fc730100f50 2024-05-08T22:36:11.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc73690c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73019ec30 con 0x7fc7301018f0 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc72ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7300ffd60 0x7fc730068c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc72ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7300ffd60 0x7fc730068c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43174/0 (socket says 172.21.15.62:43174) 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc72ffff700 1 -- 172.21.15.62:0/1927881851 learned_addr learned my addr 172.21.15.62:0/1927881851 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc72f7fe700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc730100f50 0x7fc73019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc734ea9700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7301018f0 0x7fc73019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc72ffff700 1 -- 172.21.15.62:0/1927881851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7301018f0 msgr2=0x7fc73019e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc72ffff700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7301018f0 0x7fc73019e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc72ffff700 1 -- 172.21.15.62:0/1927881851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc730100f50 msgr2=0x7fc73019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc72ffff700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc730100f50 0x7fc73019bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc72ffff700 1 -- 172.21.15.62:0/1927881851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7200095f0 con 0x7fc7300ffd60 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.790+0000 7fc72ffff700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7300ffd60 0x7fc730068c80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc720003820 tx=0x7fc720004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.791+0000 7fc72d7fa700 1 -- 172.21.15.62:0/1927881851 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72001d070 con 0x7fc7300ffd60 2024-05-08T22:36:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.791+0000 7fc72d7fa700 1 -- 172.21.15.62:0/1927881851 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc720004ca0 con 0x7fc7300ffd60 2024-05-08T22:36:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.791+0000 7fc73690c700 1 -- 172.21.15.62:0/1927881851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73019eeb0 con 0x7fc7300ffd60 2024-05-08T22:36:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.791+0000 7fc72d7fa700 1 -- 172.21.15.62:0/1927881851 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720003d30 con 0x7fc7300ffd60 2024-05-08T22:36:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.791+0000 7fc73690c700 1 -- 172.21.15.62:0/1927881851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73019f2d0 con 0x7fc7300ffd60 2024-05-08T22:36:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.792+0000 7fc73690c700 1 -- 172.21.15.62:0/1927881851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc730067d80 con 0x7fc7300ffd60 2024-05-08T22:36:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.793+0000 7fc72d7fa700 1 -- 172.21.15.62:0/1927881851 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc720005030 con 0x7fc7300ffd60 2024-05-08T22:36:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.794+0000 7fc72d7fa700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc7180a1230 0x7fc7180a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.794+0000 7fc72f7fe700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc7180a1230 0x7fc7180a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.794+0000 7fc72d7fa700 1 -- 172.21.15.62:0/1927881851 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc7200cd980 con 0x7fc7300ffd60 2024-05-08T22:36:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.794+0000 7fc72f7fe700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc7180a1230 0x7fc7180a36f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc730069240 tx=0x7fc7240073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:11.797+0000 7fc72d7fa700 1 -- 172.21.15.62:0/1927881851 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc720097110 con 0x7fc7300ffd60 2024-05-08T22:36:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:11 smithi066 ceph-mon[46830]: pgmap v254: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 7.3 KiB/s, 1 keys/s, 2 objects/s recovering 2024-05-08T22:36:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:11 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1271404888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:36:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:11 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2971129957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:36:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:11 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2669931244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:36:11.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:11 smithi079 ceph-mon[46738]: pgmap v254: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 7.3 KiB/s, 1 keys/s, 2 objects/s recovering 2024-05-08T22:36:11.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:11 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1271404888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:36:11.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:11 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2971129957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:36:11.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:11 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2669931244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:36:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.007+0000 7fc73690c700 1 -- 172.21.15.62:0/1927881851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc730067d80 con 0x7fc7300ffd60 2024-05-08T22:36:12.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.008+0000 7fc72d7fa700 1 -- 172.21.15.62:0/1927881851 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc730067d80 con 0x7fc7300ffd60 2024-05-08T22:36:12.008 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-05-08T22:36:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 -- 172.21.15.62:0/1927881851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc7180a1230 msgr2=0x7fc7180a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc7180a1230 0x7fc7180a36f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc730069240 tx=0x7fc7240073d0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 -- 172.21.15.62:0/1927881851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7300ffd60 msgr2=0x7fc730068c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7300ffd60 0x7fc730068c80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc720003820 tx=0x7fc720004730 comp rx=0 tx=0).stop 2024-05-08T22:36:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 -- 172.21.15.62:0/1927881851 shutdown_connections 2024-05-08T22:36:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7301018f0 0x7fc73019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7300ffd60 0x7fc730068c80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc7180a1230 0x7fc7180a36f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 --2- 172.21.15.62:0/1927881851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc730100f50 0x7fc73019bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 -- 172.21.15.62:0/1927881851 >> 172.21.15.62:0/1927881851 conn(0x7fc7300fb290 msgr2=0x7fc730104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 -- 172.21.15.62:0/1927881851 shutdown_connections 2024-05-08T22:36:12.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.010+0000 7fc73690c700 1 -- 172.21.15.62:0/1927881851 wait complete. 2024-05-08T22:36:12.022 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:36:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.152+0000 7fad7f19c700 1 -- 172.21.15.62:0/640357208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad780ffda0 msgr2=0x7fad781001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.152+0000 7fad7f19c700 1 --2- 172.21.15.62:0/640357208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad780ffda0 0x7fad781001c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0099c0 tx=0x7fad6c009cd0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 -- 172.21.15.62:0/640357208 shutdown_connections 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 --2- 172.21.15.62:0/640357208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78101930 0x7fad781081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 --2- 172.21.15.62:0/640357208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad780ffda0 0x7fad781001c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 --2- 172.21.15.62:0/640357208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad78100f90 0x7fad781013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 -- 172.21.15.62:0/640357208 >> 172.21.15.62:0/640357208 conn(0x7fad780fb330 msgr2=0x7fad780fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 -- 172.21.15.62:0/640357208 shutdown_connections 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 -- 172.21.15.62:0/640357208 wait complete. 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 Processor -- start 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 -- start start 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad780ffda0 0x7fad78196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78100f90 0x7fad78197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad78101930 0x7fad7819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7819e9a0 con 0x7fad780ffda0 2024-05-08T22:36:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7819eae0 con 0x7fad78101930 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7f19c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7819ec70 con 0x7fad78100f90 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7cf38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad780ffda0 0x7fad78196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7cf38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad780ffda0 0x7fad78196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43198/0 (socket says 172.21.15.62:43198) 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.153+0000 7fad7cf38700 1 -- 172.21.15.62:0/2012449825 learned_addr learned my addr 172.21.15.62:0/2012449825 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad77fff700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78100f90 0x7fad78197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad7d739700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad78101930 0x7fad7819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad7cf38700 1 -- 172.21.15.62:0/2012449825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78100f90 msgr2=0x7fad78197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad7cf38700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78100f90 0x7fad78197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad7cf38700 1 -- 172.21.15.62:0/2012449825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad78101930 msgr2=0x7fad7819e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad7cf38700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad78101930 0x7fad7819e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad7cf38700 1 -- 172.21.15.62:0/2012449825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c0095f0 con 0x7fad780ffda0 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad7d739700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad78101930 0x7fad7819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad77fff700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78100f90 0x7fad78197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad7cf38700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad780ffda0 0x7fad78196ec0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fad6c006010 tx=0x7fad6c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad75ffb700 1 -- 172.21.15.62:0/2012449825 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c01d070 con 0x7fad780ffda0 2024-05-08T22:36:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad75ffb700 1 -- 172.21.15.62:0/2012449825 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad6c0054e0 con 0x7fad780ffda0 2024-05-08T22:36:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad75ffb700 1 -- 172.21.15.62:0/2012449825 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004c10 con 0x7fad780ffda0 2024-05-08T22:36:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad7f19c700 1 -- 172.21.15.62:0/2012449825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7819eef0 con 0x7fad780ffda0 2024-05-08T22:36:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.154+0000 7fad7f19c700 1 -- 172.21.15.62:0/2012449825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7819f260 con 0x7fad780ffda0 2024-05-08T22:36:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.155+0000 7fad7f19c700 1 -- 172.21.15.62:0/2012449825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad780681d0 con 0x7fad780ffda0 2024-05-08T22:36:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.156+0000 7fad75ffb700 1 -- 172.21.15.62:0/2012449825 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fad6c00fbb0 con 0x7fad780ffda0 2024-05-08T22:36:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.157+0000 7fad75ffb700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fad680a11b0 0x7fad680a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.157+0000 7fad77fff700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fad680a11b0 0x7fad680a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.157+0000 7fad75ffb700 1 -- 172.21.15.62:0/2012449825 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fad6c0cd630 con 0x7fad780ffda0 2024-05-08T22:36:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.157+0000 7fad77fff700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fad680a11b0 0x7fad680a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad64009cb0 tx=0x7fad64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.161+0000 7fad75ffb700 1 -- 172.21.15.62:0/2012449825 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fad780681d0 con 0x7fad780ffda0 2024-05-08T22:36:12.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.369+0000 7fad7f19c700 1 -- 172.21.15.62:0/2012449825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad7819f3f0 con 0x7fad780ffda0 2024-05-08T22:36:12.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.369+0000 7fad75ffb700 1 -- 172.21.15.62:0/2012449825 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7fad6c096ca0 con 0x7fad780ffda0 2024-05-08T22:36:12.369 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-05-08T22:36:12.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 -- 172.21.15.62:0/2012449825 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fad680a11b0 msgr2=0x7fad680a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fad680a11b0 0x7fad680a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad64009cb0 tx=0x7fad64009260 comp rx=0 tx=0).stop 2024-05-08T22:36:12.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 -- 172.21.15.62:0/2012449825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad780ffda0 msgr2=0x7fad78196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad780ffda0 0x7fad78196ec0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fad6c006010 tx=0x7fad6c00f740 comp rx=0 tx=0).stop 2024-05-08T22:36:12.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 -- 172.21.15.62:0/2012449825 shutdown_connections 2024-05-08T22:36:12.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fad680a11b0 0x7fad680a3670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad78101930 0x7fad7819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad780ffda0 0x7fad78196ec0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 --2- 172.21.15.62:0/2012449825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78100f90 0x7fad78197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 -- 172.21.15.62:0/2012449825 >> 172.21.15.62:0/2012449825 conn(0x7fad780fb330 msgr2=0x7fad78069470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 -- 172.21.15.62:0/2012449825 shutdown_connections 2024-05-08T22:36:12.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.372+0000 7fad7f19c700 1 -- 172.21.15.62:0/2012449825 wait complete. 2024-05-08T22:36:12.384 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:36:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.511+0000 7fdc1af55700 1 -- 172.21.15.62:0/2790576470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc14100e80 msgr2=0x7fdc141012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.511+0000 7fdc1af55700 1 --2- 172.21.15.62:0/2790576470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc14100e80 0x7fdc141012a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009990 tx=0x7fdc08009ca0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.511+0000 7fdc1af55700 1 -- 172.21.15.62:0/2790576470 shutdown_connections 2024-05-08T22:36:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.511+0000 7fdc1af55700 1 --2- 172.21.15.62:0/2790576470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc14102990 0x7fdc141092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.511+0000 7fdc1af55700 1 --2- 172.21.15.62:0/2790576470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc14100e80 0x7fdc141012a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.511+0000 7fdc1af55700 1 --2- 172.21.15.62:0/2790576470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc14101fd0 0x7fdc14102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.511+0000 7fdc1af55700 1 -- 172.21.15.62:0/2790576470 >> 172.21.15.62:0/2790576470 conn(0x7fdc140fc410 msgr2=0x7fdc140fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.511+0000 7fdc1af55700 1 -- 172.21.15.62:0/2790576470 shutdown_connections 2024-05-08T22:36:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.511+0000 7fdc1af55700 1 -- 172.21.15.62:0/2790576470 wait complete. 2024-05-08T22:36:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc1af55700 1 Processor -- start 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc1af55700 1 -- start start 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc1af55700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc14100e80 0x7fdc14196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc1af55700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc14101fd0 0x7fdc14197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc1af55700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc14102990 0x7fdc1419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc1af55700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1419e950 con 0x7fdc14101fd0 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc1af55700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1419ea90 con 0x7fdc14102990 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc1af55700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1419ec20 con 0x7fdc14100e80 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc13fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc14101fd0 0x7fdc14197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc13fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc14101fd0 0x7fdc14197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43216/0 (socket says 172.21.15.62:43216) 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc13fff700 1 -- 172.21.15.62:0/3033470834 learned_addr learned my addr 172.21.15.62:0/3033470834 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc18cf1700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc14100e80 0x7fdc14196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.512+0000 7fdc194f2700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc14102990 0x7fdc1419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.513+0000 7fdc13fff700 1 -- 172.21.15.62:0/3033470834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc14100e80 msgr2=0x7fdc14196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.513+0000 7fdc13fff700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc14100e80 0x7fdc14196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.513+0000 7fdc13fff700 1 -- 172.21.15.62:0/3033470834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc14102990 msgr2=0x7fdc1419e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.513+0000 7fdc13fff700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc14102990 0x7fdc1419e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.513+0000 7fdc13fff700 1 -- 172.21.15.62:0/3033470834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc080095f0 con 0x7fdc14101fd0 2024-05-08T22:36:12.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.513+0000 7fdc13fff700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc14101fd0 0x7fdc14197360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000d950 tx=0x7fdc0000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.513+0000 7fdc11ffb700 1 -- 172.21.15.62:0/3033470834 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000f890 con 0x7fdc14101fd0 2024-05-08T22:36:12.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.513+0000 7fdc11ffb700 1 -- 172.21.15.62:0/3033470834 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc00010460 con 0x7fdc14101fd0 2024-05-08T22:36:12.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.513+0000 7fdc1af55700 1 -- 172.21.15.62:0/3033470834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1419ef00 con 0x7fdc14101fd0 2024-05-08T22:36:12.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.513+0000 7fdc11ffb700 1 -- 172.21.15.62:0/3033470834 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000e610 con 0x7fdc14101fd0 2024-05-08T22:36:12.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.513+0000 7fdc1af55700 1 -- 172.21.15.62:0/3033470834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1419f380 con 0x7fdc14101fd0 2024-05-08T22:36:12.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.514+0000 7fdc1af55700 1 -- 172.21.15.62:0/3033470834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc141082f0 con 0x7fdc14101fd0 2024-05-08T22:36:12.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.515+0000 7fdc11ffb700 1 -- 172.21.15.62:0/3033470834 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdc000108d0 con 0x7fdc14101fd0 2024-05-08T22:36:12.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.516+0000 7fdc11ffb700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc040a11b0 0x7fdc040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.516+0000 7fdc11ffb700 1 -- 172.21.15.62:0/3033470834 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdc000cc7f0 con 0x7fdc14101fd0 2024-05-08T22:36:12.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.516+0000 7fdc18cf1700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc040a11b0 0x7fdc040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.516+0000 7fdc18cf1700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc040a11b0 0x7fdc040a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800b5c0 tx=0x7fdc08003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.520+0000 7fdc11ffb700 1 -- 172.21.15.62:0/3033470834 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdc00095e30 con 0x7fdc14101fd0 2024-05-08T22:36:12.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:12 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/216602812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:36:12.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:12 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1927881851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:36:12.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:12 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2012449825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:36:12.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.773+0000 7fdc1af55700 1 -- 172.21.15.62:0/3033470834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc14067480 con 0x7fdc14101fd0 2024-05-08T22:36:12.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.774+0000 7fdc11ffb700 1 -- 172.21.15.62:0/3033470834 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10897 (secure 0 0 0) 0x7fdc00099450 con 0x7fdc14101fd0 2024-05-08T22:36:12.774 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:36:12.774 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:35:09.803459+0000","modified":"2024-05-08T22:35:16.261557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24529,"mds_2":24505,"mds_3":24503,"mds_4":14661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/1756743060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1756743060},{"type":"v1","addr":"172.21.15.62:6839","nonce":1756743060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14661":{"gid":14661,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/780017086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":780017086},{"type":"v1","addr":"172.21.15.62:6841","nonce":780017086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.79:6839/3534785027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":3534785027},{"type":"v1","addr":"172.21.15.79:6839","nonce":3534785027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/3474672912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":3474672912},{"type":"v1","addr":"172.21.15.66:6839","nonce":3474672912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:36:12.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.776+0000 7fdc1af55700 1 -- 172.21.15.62:0/3033470834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc040a11b0 msgr2=0x7fdc040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.776+0000 7fdc1af55700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc040a11b0 0x7fdc040a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800b5c0 tx=0x7fdc08003790 comp rx=0 tx=0).stop 2024-05-08T22:36:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.776+0000 7fdc1af55700 1 -- 172.21.15.62:0/3033470834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc14101fd0 msgr2=0x7fdc14197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.776+0000 7fdc1af55700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc14101fd0 0x7fdc14197360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000d950 tx=0x7fdc0000dc60 comp rx=0 tx=0).stop 2024-05-08T22:36:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.776+0000 7fdc1af55700 1 -- 172.21.15.62:0/3033470834 shutdown_connections 2024-05-08T22:36:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.776+0000 7fdc1af55700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fdc040a11b0 0x7fdc040a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.776+0000 7fdc1af55700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc14102990 0x7fdc1419e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.776+0000 7fdc1af55700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc14100e80 0x7fdc14196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.776+0000 7fdc1af55700 1 --2- 172.21.15.62:0/3033470834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc14101fd0 0x7fdc14197360 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.777+0000 7fdc1af55700 1 -- 172.21.15.62:0/3033470834 >> 172.21.15.62:0/3033470834 conn(0x7fdc140fc410 msgr2=0x7fdc14105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.777+0000 7fdc1af55700 1 -- 172.21.15.62:0/3033470834 shutdown_connections 2024-05-08T22:36:12.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.777+0000 7fdc1af55700 1 -- 172.21.15.62:0/3033470834 wait complete. 2024-05-08T22:36:12.778 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-05-08T22:36:12.789 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:36:12.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:12 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/216602812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:36:12.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:12 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1927881851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:36:12.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:12 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2012449825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:36:12.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:12 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/216602812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:36:12.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:12 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1927881851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:36:12.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:12 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2012449825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:36:12.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.915+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/3277645738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ec100f70 msgr2=0x7fd8ec1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.915+0000 7fd8f3e2c700 1 --2- 172.21.15.62:0/3277645738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ec100f70 0x7fd8ec1013d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0099c0 tx=0x7fd8dc009cd0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.915+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/3277645738 shutdown_connections 2024-05-08T22:36:12.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.915+0000 7fd8f3e2c700 1 --2- 172.21.15.62:0/3277645738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8ec101910 0x7fd8ec1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.915+0000 7fd8f3e2c700 1 --2- 172.21.15.62:0/3277645738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec0ffd80 0x7fd8ec1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.915+0000 7fd8f3e2c700 1 --2- 172.21.15.62:0/3277645738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ec100f70 0x7fd8ec1013d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.915+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/3277645738 >> 172.21.15.62:0/3277645738 conn(0x7fd8ec0fb310 msgr2=0x7fd8ec0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.915+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/3277645738 shutdown_connections 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.915+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/3277645738 wait complete. 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f3e2c700 1 Processor -- start 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f3e2c700 1 -- start start 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f3e2c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ec0ffd80 0x7fd8ec068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f3e2c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec100f70 0x7fd8ec19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f3e2c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8ec101910 0x7fd8ec19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f3e2c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec19e8a0 con 0x7fd8ec0ffd80 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f3e2c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec19ea10 con 0x7fd8ec100f70 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f3e2c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec19ebd0 con 0x7fd8ec101910 2024-05-08T22:36:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f1bc8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ec0ffd80 0x7fd8ec068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f1bc8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ec0ffd80 0x7fd8ec068ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43228/0 (socket says 172.21.15.62:43228) 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f1bc8700 1 -- 172.21.15.62:0/125675125 learned_addr learned my addr 172.21.15.62:0/125675125 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f13c7700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec100f70 0x7fd8ec19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f23c9700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8ec101910 0x7fd8ec19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f1bc8700 1 -- 172.21.15.62:0/125675125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8ec101910 msgr2=0x7fd8ec19e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f1bc8700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8ec101910 0x7fd8ec19e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f1bc8700 1 -- 172.21.15.62:0/125675125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec100f70 msgr2=0x7fd8ec19ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f1bc8700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec100f70 0x7fd8ec19ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f1bc8700 1 -- 172.21.15.62:0/125675125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0095f0 con 0x7fd8ec0ffd80 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.916+0000 7fd8f23c9700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8ec101910 0x7fd8ec19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.917+0000 7fd8f1bc8700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ec0ffd80 0x7fd8ec068ca0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e970 tx=0x7fd8e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.917+0000 7fd8e2ffd700 1 -- 172.21.15.62:0/125675125 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8009790 con 0x7fd8ec0ffd80 2024-05-08T22:36:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.917+0000 7fd8e2ffd700 1 -- 172.21.15.62:0/125675125 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8e800eec0 con 0x7fd8ec0ffd80 2024-05-08T22:36:12.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.917+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/125675125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec19ee50 con 0x7fd8ec0ffd80 2024-05-08T22:36:12.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.917+0000 7fd8e2ffd700 1 -- 172.21.15.62:0/125675125 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8003cf0 con 0x7fd8ec0ffd80 2024-05-08T22:36:12.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.917+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/125675125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec19f280 con 0x7fd8ec0ffd80 2024-05-08T22:36:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.918+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/125675125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ec107130 con 0x7fd8ec0ffd80 2024-05-08T22:36:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.919+0000 7fd8e2ffd700 1 -- 172.21.15.62:0/125675125 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd8e800ccd0 con 0x7fd8ec0ffd80 2024-05-08T22:36:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.920+0000 7fd8e2ffd700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8d80a1230 0x7fd8d80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.920+0000 7fd8f13c7700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8d80a1230 0x7fd8d80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.920+0000 7fd8e2ffd700 1 -- 172.21.15.62:0/125675125 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd8e8014070 con 0x7fd8ec0ffd80 2024-05-08T22:36:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.921+0000 7fd8f13c7700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8d80a1230 0x7fd8d80a36f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc006010 tx=0x7fd8dc00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:12.923+0000 7fd8e2ffd700 1 -- 172.21.15.62:0/125675125 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd8e8096400 con 0x7fd8ec0ffd80 2024-05-08T22:36:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.132+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/125675125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8ec067d10 con 0x7fd8ec0ffd80 2024-05-08T22:36:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.133+0000 7fd8e2ffd700 1 -- 172.21.15.62:0/125675125 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22481 (secure 0 0 0) 0x7fd8e8099a20 con 0x7fd8ec0ffd80 2024-05-08T22:36:13.133 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:36:13.134 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:36:05.960694+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:36:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/125675125 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8d80a1230 msgr2=0x7fd8d80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8d80a1230 0x7fd8d80a36f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc006010 tx=0x7fd8dc00b540 comp rx=0 tx=0).stop 2024-05-08T22:36:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/125675125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ec0ffd80 msgr2=0x7fd8ec068ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ec0ffd80 0x7fd8ec068ca0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e970 tx=0x7fd8e800ec80 comp rx=0 tx=0).stop 2024-05-08T22:36:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/125675125 shutdown_connections 2024-05-08T22:36:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8ec101910 0x7fd8ec19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ec0ffd80 0x7fd8ec068ca0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fd8d80a1230 0x7fd8d80a36f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 --2- 172.21.15.62:0/125675125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8ec100f70 0x7fd8ec19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/125675125 >> 172.21.15.62:0/125675125 conn(0x7fd8ec0fb310 msgr2=0x7fd8ec104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/125675125 shutdown_connections 2024-05-08T22:36:13.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.136+0000 7fd8f3e2c700 1 -- 172.21.15.62:0/125675125 wait complete. 2024-05-08T22:36:13.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T22:35:09.803459+0000', 'modified': '2024-05-08T22:35:16.261557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24533, 'mds_1': 24529, 'mds_2': 24505, 'mds_3': 24503, 'mds_4': 14661}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14649': {'gid': 14649, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6839/1756743060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 1756743060}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 1756743060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14661': {'gid': 14661, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.62:6841/780017086', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6840', 'nonce': 780017086}, {'type': 'v1', 'addr': '172.21.15.62:6841', 'nonce': 780017086}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24455': {'gid': 24455, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6833/4065089721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 4065089721}, {'type': 'v1', 'addr': '172.21.15.66:6833', 'nonce': 4065089721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24466': {'gid': 24466, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6833/1379725080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 1379725080}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 1379725080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6835/2449812673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 2449812673}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 2449812673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6835/2938225588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 2938225588}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 2938225588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24503': {'gid': 24503, 'name': 'h', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.66:6837/1178845415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 1178845415}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 1178845415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.79:6837/753714978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 753714978}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 753714978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.79:6839/3534785027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 3534785027}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 3534785027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24533': {'gid': 24533, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6839/3474672912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 3474672912}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 3474672912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T22:36:13.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-08T22:36:13.149 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T22:36:13.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.276+0000 7f60bf69d700 1 -- 172.21.15.62:0/2578493130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60b81018d0 msgr2=0x7f60b8108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.276+0000 7f60bf69d700 1 --2- 172.21.15.62:0/2578493130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60b81018d0 0x7f60b8108180 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f60b000b0d0 tx=0x7f60b000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 -- 172.21.15.62:0/2578493130 shutdown_connections 2024-05-08T22:36:13.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 --2- 172.21.15.62:0/2578493130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60b81018d0 0x7f60b8108180 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 --2- 172.21.15.62:0/2578493130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b80ffd40 0x7f60b8100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 --2- 172.21.15.62:0/2578493130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b8100f30 0x7f60b8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 -- 172.21.15.62:0/2578493130 >> 172.21.15.62:0/2578493130 conn(0x7f60b80fb2b0 msgr2=0x7f60b80fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 -- 172.21.15.62:0/2578493130 shutdown_connections 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 -- 172.21.15.62:0/2578493130 wait complete. 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 Processor -- start 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 -- start start 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80ffd40 0x7f60b8196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60b8100f30 0x7f60b81973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.277+0000 7f60bf69d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b81018d0 0x7f60b819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bf69d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b819e9b0 con 0x7f60b8100f30 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bf69d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b819eaf0 con 0x7f60b81018d0 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bf69d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b819ec80 con 0x7f60b80ffd40 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bcc38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60b8100f30 0x7f60b81973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bcc38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60b8100f30 0x7f60b81973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43240/0 (socket says 172.21.15.62:43240) 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bcc38700 1 -- 172.21.15.62:0/2157176814 learned_addr learned my addr 172.21.15.62:0/2157176814 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bd439700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80ffd40 0x7f60b8196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bdc3a700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b81018d0 0x7f60b819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bcc38700 1 -- 172.21.15.62:0/2157176814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80ffd40 msgr2=0x7f60b8196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bcc38700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80ffd40 0x7f60b8196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bcc38700 1 -- 172.21.15.62:0/2157176814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b81018d0 msgr2=0x7f60b819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bcc38700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b81018d0 0x7f60b819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bcc38700 1 -- 172.21.15.62:0/2157176814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b0009d00 con 0x7f60b8100f30 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bcc38700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60b8100f30 0x7f60b81973c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d950 tx=0x7f60a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60ae7fc700 1 -- 172.21.15.62:0/2157176814 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a800f890 con 0x7f60b8100f30 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60ae7fc700 1 -- 172.21.15.62:0/2157176814 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a8010460 con 0x7f60b8100f30 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bf69d700 1 -- 172.21.15.62:0/2157176814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b819ef60 con 0x7f60b8100f30 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60ae7fc700 1 -- 172.21.15.62:0/2157176814 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a800e610 con 0x7f60b8100f30 2024-05-08T22:36:13.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.278+0000 7f60bf69d700 1 -- 172.21.15.62:0/2157176814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b819f430 con 0x7f60b8100f30 2024-05-08T22:36:13.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.280+0000 7f60ae7fc700 1 -- 172.21.15.62:0/2157176814 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f60a80108d0 con 0x7f60b8100f30 2024-05-08T22:36:13.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.281+0000 7f60ae7fc700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60a40a1230 0x7f60a40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.281+0000 7f60bd439700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60a40a1230 0x7f60a40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.281+0000 7f60ae7fc700 1 -- 172.21.15.62:0/2157176814 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f60a80cc780 con 0x7f60b8100f30 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.281+0000 7f60bd439700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60a40a1230 0x7f60a40a36f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f60b8100d90 tx=0x7f60b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60cb857b80 1 Processor -- start 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60cb857b80 1 -- start start 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 0x562b74106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b741072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74107810 0x562b74107c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60cb857b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7405d100 con 0x562b74106e50 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60cb857b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7405aac0 con 0x562b74107810 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60cb857b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b740563b0 con 0x562b741064f0 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60bcc38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b741072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60bcc38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b741072d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43244/0 (socket says 172.21.15.62:43244) 2024-05-08T22:36:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60bcc38700 1 -- 172.21.15.62:0/2706664009 learned_addr learned my addr 172.21.15.62:0/2706664009 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.282+0000 7f60bd439700 1 --2- 172.21.15.62:0/2706664009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 0x562b74106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.283+0000 7f60bdc3a700 1 --2- 172.21.15.62:0/2706664009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74107810 0x562b74107c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.283+0000 7f60bcc38700 1 -- 172.21.15.62:0/2706664009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 msgr2=0x562b74106910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.283+0000 7f60bcc38700 1 --2- 172.21.15.62:0/2706664009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 0x562b74106910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.283+0000 7f60bcc38700 1 -- 172.21.15.62:0/2706664009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74107810 msgr2=0x562b74107c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.283+0000 7f60bcc38700 1 --2- 172.21.15.62:0/2706664009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74107810 0x562b74107c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.283+0000 7f60bcc38700 1 -- 172.21.15.62:0/2706664009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b73ed7360 con 0x562b74106e50 2024-05-08T22:36:13.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.283+0000 7f60bcc38700 1 --2- 172.21.15.62:0/2706664009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b741072d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f60a80a19d0 tx=0x7f60a809a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2df398c8f23162 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.283+0000 7f60a37fe700 1 -- 172.21.15.62:0/2706664009 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a80a1f00 con 0x562b74106e50 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.283+0000 7f60a37fe700 1 -- 172.21.15.62:0/2706664009 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a80ca390 con 0x562b74106e50 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.283+0000 7f60a37fe700 1 -- 172.21.15.62:0/2706664009 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a80991b0 con 0x562b74106e50 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.284+0000 7f60cb857b80 1 -- 172.21.15.62:0/2706664009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 msgr2=0x562b741072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.284+0000 7f60cb857b80 1 --2- 172.21.15.62:0/2706664009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b741072d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f60a80a19d0 tx=0x7f60a809a180 comp rx=0 tx=0).stop 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.284+0000 7f60cb857b80 1 -- 172.21.15.62:0/2706664009 shutdown_connections 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.284+0000 7f60cb857b80 1 --2- 172.21.15.62:0/2706664009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74107810 0x562b74107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.284+0000 7f60cb857b80 1 --2- 172.21.15.62:0/2706664009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 0x562b74106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.284+0000 7f60cb857b80 1 --2- 172.21.15.62:0/2706664009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b741072d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.284+0000 7f60cb857b80 1 -- 172.21.15.62:0/2706664009 >> 172.21.15.62:0/2706664009 conn(0x562b740641c0 msgr2=0x562b74100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.284+0000 7f60cb857b80 1 -- 172.21.15.62:0/2706664009 shutdown_connections 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.284+0000 7f60cb857b80 1 -- 172.21.15.62:0/2706664009 wait complete. 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.284+0000 7f60cb857b80 1 -- start start 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.284+0000 7f60cb857b80 1 Processor -- start 2024-05-08T22:36:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b741064f0 0x562b7410fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b74106e50 0x562b741102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74117300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60cb857b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b73ff2510 con 0x562b74107810 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60cb857b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b73ff21b0 con 0x562b741064f0 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60cb857b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b73ed9650 con 0x562b74106e50 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60cb857b80 10 client.0 _ll_register_callbacks cb 0x562b73ffbae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bdc3a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74117300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bd439700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b741064f0 0x562b7410fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bdc3a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74117300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43250/0 (socket says 172.21.15.62:43250) 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bdc3a700 1 -- 172.21.15.62:0/1759429446 learned_addr learned my addr 172.21.15.62:0/1759429446 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bcc38700 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b74106e50 0x562b741102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bdc3a700 1 -- 172.21.15.62:0/1759429446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b74106e50 msgr2=0x562b741102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bdc3a700 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b74106e50 0x562b741102c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bdc3a700 1 -- 172.21.15.62:0/1759429446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b741064f0 msgr2=0x562b7410fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bdc3a700 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b741064f0 0x562b7410fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bdc3a700 1 -- 172.21.15.62:0/1759429446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b819f430 con 0x562b74107810 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bd439700 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b741064f0 0x562b7410fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60bdc3a700 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74117300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f60b000b0d0 tx=0x7f60b0009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245489ddc2fcdb17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1759429446 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b000e030 con 0x562b74107810 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1759429446 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f60b0003c10 con 0x562b74107810 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.285+0000 7f60cb857b80 1 -- 172.21.15.62:0/1759429446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b73faf220 con 0x562b74107810 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1759429446 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b0003e10 con 0x562b74107810 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60cb857b80 10 client.15279 fetch_fsmap learned FSMap version 26 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60cb857b80 1 -- 172.21.15.62:0/1759429446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562b73e83b10 con 0x562b74107810 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1759429446 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562b73faf220 con 0x562b74107810 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1759429446 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f60b002e070 con 0x562b74107810 2024-05-08T22:36:13.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60cb857b80 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60cb857b80 10 client.15279 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60cb857b80 20 client.15279 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x562b7411aff0 0x562b741202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60cb857b80 4 client.15279 mds_command: new command op to 14661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60cb857b80 1 -- 172.21.15.62:0/1759429446 --> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b740d57c0 con 0x562b7411aff0 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60bd439700 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x562b7411aff0 0x562b741202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60a0ff9700 10 client.15279 ms_handle_connect on v2:172.21.15.62:6840/780017086 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.286+0000 7f60bd439700 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x562b7411aff0 0x562b741202d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6dadb50797dae36a server_cookie=5e4f4c93d71d461e in_seq=0 out_seq=0 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1759429446 <== mds.4 v2:172.21.15.62:6840/780017086 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b740d57c0 con 0x562b7411aff0 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60a0ff9700 10 client.15279 handle_command_reply: tid=0 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 1 client.15279 shutdown 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 1 -- 172.21.15.62:0/1759429446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 msgr2=0x562b74117300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74117300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f60b000b0d0 tx=0x7f60b0009c90 comp rx=0 tx=0).stop 2024-05-08T22:36:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 1 -- 172.21.15.62:0/1759429446 shutdown_connections 2024-05-08T22:36:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74117300 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x562b7411aff0 0x562b741202d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b741064f0 0x562b7410fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1759429446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b74106e50 0x562b741102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60a0ff9700 0 client.15279 ms_handle_reset on v2:172.21.15.62:6840/780017086 2024-05-08T22:36:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 1 -- 172.21.15.62:0/1759429446 >> 172.21.15.62:0/1759429446 conn(0x562b740641c0 msgr2=0x562b74100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 1 -- 172.21.15.62:0/1759429446 shutdown_connections 2024-05-08T22:36:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 1 -- 172.21.15.62:0/1759429446 wait complete. 2024-05-08T22:36:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.287+0000 7f60cb857b80 20 client.15279 trim_cache size 0 max 16384 2024-05-08T22:36:13.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60cb857b80 1 Processor -- start 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60cb857b80 1 -- start start 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 0x562b741068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b7410dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74107810 0x562b7410f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60cb857b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b73faf250 con 0x562b74106e50 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60cb857b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7404f180 con 0x562b74107810 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60cb857b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b73ff55a0 con 0x562b741064f0 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60bcc38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b7410dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60bcc38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b7410dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43256/0 (socket says 172.21.15.62:43256) 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60bcc38700 1 -- 172.21.15.62:0/1912606678 learned_addr learned my addr 172.21.15.62:0/1912606678 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60bdc3a700 1 --2- 172.21.15.62:0/1912606678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74107810 0x562b7410f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60bcc38700 1 -- 172.21.15.62:0/1912606678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 msgr2=0x562b741068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60bd439700 1 --2- 172.21.15.62:0/1912606678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 0x562b741068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60bcc38700 1 --2- 172.21.15.62:0/1912606678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 0x562b741068d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60bcc38700 1 -- 172.21.15.62:0/1912606678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74107810 msgr2=0x562b7410f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60bcc38700 1 --2- 172.21.15.62:0/1912606678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74107810 0x562b7410f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60bcc38700 1 -- 172.21.15.62:0/1912606678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b7401c510 con 0x562b74106e50 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.306+0000 7f60bcc38700 1 --2- 172.21.15.62:0/1912606678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b7410dd10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f60a8097dd0 tx=0x7f60a80cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8543fbeecf9432ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60a3fff700 1 -- 172.21.15.62:0/1912606678 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a80a1f00 con 0x562b74106e50 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60a3fff700 1 -- 172.21.15.62:0/1912606678 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a80b2240 con 0x562b74106e50 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60a3fff700 1 -- 172.21.15.62:0/1912606678 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8099f40 con 0x562b74106e50 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60cb857b80 1 -- 172.21.15.62:0/1912606678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 msgr2=0x562b7410dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1912606678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b7410dd10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f60a8097dd0 tx=0x7f60a80cc040 comp rx=0 tx=0).stop 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60cb857b80 1 -- 172.21.15.62:0/1912606678 shutdown_connections 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1912606678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74107810 0x562b7410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1912606678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 0x562b741068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1912606678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74106e50 0x562b7410dd10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60cb857b80 1 -- 172.21.15.62:0/1912606678 >> 172.21.15.62:0/1912606678 conn(0x562b740641c0 msgr2=0x562b74107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60cb857b80 1 -- 172.21.15.62:0/1912606678 shutdown_connections 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.307+0000 7f60cb857b80 1 -- 172.21.15.62:0/1912606678 wait complete. 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60cb857b80 1 -- start start 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60cb857b80 1 Processor -- start 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 0x562b7410cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74106e50 0x562b740ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60cb857b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74100200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60cb857b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7410d530 con 0x562b74107810 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60cb857b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b74100740 con 0x562b74106e50 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60cb857b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b74100900 con 0x562b741064f0 2024-05-08T22:36:13.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60cb857b80 10 client.0 _ll_register_callbacks cb 0x562b73ffbae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:13.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60bdc3a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74100200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60bdc3a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74100200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43272/0 (socket says 172.21.15.62:43272) 2024-05-08T22:36:13.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60bdc3a700 1 -- 172.21.15.62:0/1254318696 learned_addr learned my addr 172.21.15.62:0/1254318696 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60bcc38700 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74106e50 0x562b740ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60bdc3a700 1 -- 172.21.15.62:0/1254318696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 msgr2=0x562b7410cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60bdc3a700 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 0x562b7410cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60bdc3a700 1 -- 172.21.15.62:0/1254318696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74106e50 msgr2=0x562b740ffcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60bdc3a700 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74106e50 0x562b740ffcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.308+0000 7f60bdc3a700 1 -- 172.21.15.62:0/1254318696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b73ed7360 con 0x562b74107810 2024-05-08T22:36:13.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60bdc3a700 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74100200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f60b000b420 tx=0x7f60b0006220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd51413d4de32a41 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:13.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1254318696 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b000e070 con 0x562b74107810 2024-05-08T22:36:13.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1254318696 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f60b001dda0 con 0x562b74107810 2024-05-08T22:36:13.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60cb857b80 1 -- 172.21.15.62:0/1254318696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60a80ca390 con 0x562b74107810 2024-05-08T22:36:13.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1254318696 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b0030450 con 0x562b74107810 2024-05-08T22:36:13.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60cb857b80 10 client.15285 fetch_fsmap learned FSMap version 26 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60cb857b80 1 -- 172.21.15.62:0/1254318696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562b73edbc80 con 0x562b74107810 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1254318696 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a80ca390 con 0x562b74107810 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1254318696 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f60b002e070 con 0x562b74107810 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60cb857b80 10 client.15285 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60cb857b80 10 client.15285 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60cb857b80 20 client.15285 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x562b7417a610 0x562b74176f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60cb857b80 4 client.15285 mds_command: new command op to 14661 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60cb857b80 1 -- 172.21.15.62:0/1254318696 --> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562b7404ee70 con 0x562b7417a610 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.309+0000 7f60bd439700 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x562b7417a610 0x562b74176f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60bd439700 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x562b7417a610 0x562b74176f20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aae6d7b1071ca8a8 server_cookie=ff2981497f2bd39d in_seq=0 out_seq=0 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60a0ff9700 10 client.15285 ms_handle_connect on v2:172.21.15.62:6840/780017086 2024-05-08T22:36:13.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60a0ff9700 1 -- 172.21.15.62:0/1254318696 <== mds.4 v2:172.21.15.62:6840/780017086 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562b7404ee70 con 0x562b7417a610 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60a0ff9700 10 client.15285 handle_command_reply: tid=0 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 1 client.15285 shutdown 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 1 -- 172.21.15.62:0/1254318696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 msgr2=0x562b74100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74100200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f60b000b420 tx=0x7f60b0006220 comp rx=0 tx=0).stop 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 1 -- 172.21.15.62:0/1254318696 shutdown_connections 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b74107810 0x562b74100200 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b741064f0 0x562b7410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.62:6840/780017086,v1:172.21.15.62:6841/780017086] conn(0x562b7417a610 0x562b74176f20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 1 --2- 172.21.15.62:0/1254318696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b74106e50 0x562b740ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60a0ff9700 0 client.15285 ms_handle_reset on v2:172.21.15.62:6840/780017086 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 1 -- 172.21.15.62:0/1254318696 >> 172.21.15.62:0/1254318696 conn(0x562b740641c0 msgr2=0x562b7410f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 1 -- 172.21.15.62:0/1254318696 shutdown_connections 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 1 -- 172.21.15.62:0/1254318696 wait complete. 2024-05-08T22:36:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.310+0000 7f60cb857b80 20 client.15285 trim_cache size 0 max 16384 2024-05-08T22:36:13.318 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:36:13.318 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:36:13.318 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 4, 2024-05-08T22:36:13.318 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 14661, 2024-05-08T22:36:13.318 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-05-08T22:36:13.318 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-05-08T22:36:13.318 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-05-08T22:36:13.318 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 58.054486038999997, 2024-05-08T22:36:13.318 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 26, 2024-05-08T22:36:13.319 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 81, 2024-05-08T22:36:13.319 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T22:36:13.319 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 71.047439793999999 2024-05-08T22:36:13.319 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:36:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 -- 172.21.15.62:0/2157176814 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60a40a1230 msgr2=0x7f60a40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60a40a1230 0x7f60a40a36f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f60b8100d90 tx=0x7f60b400b410 comp rx=0 tx=0).stop 2024-05-08T22:36:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 -- 172.21.15.62:0/2157176814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60b8100f30 msgr2=0x7f60b81973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60b8100f30 0x7f60b81973c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d950 tx=0x7f60a800dc60 comp rx=0 tx=0).stop 2024-05-08T22:36:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 -- 172.21.15.62:0/2157176814 shutdown_connections 2024-05-08T22:36:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60a40a1230 0x7f60a40a36f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b81018d0 0x7f60b819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80ffd40 0x7f60b8196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 --2- 172.21.15.62:0/2157176814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60b8100f30 0x7f60b81973c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 -- 172.21.15.62:0/2157176814 >> 172.21.15.62:0/2157176814 conn(0x7f60b80fb2b0 msgr2=0x7f60b80fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 -- 172.21.15.62:0/2157176814 shutdown_connections 2024-05-08T22:36:13.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.311+0000 7f60bf69d700 1 -- 172.21.15.62:0/2157176814 wait complete. 2024-05-08T22:36:13.323 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-08T22:36:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.453+0000 7f4621c1f700 1 -- 172.21.15.62:0/488103890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f461c101950 msgr2=0x7f461c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.453+0000 7f4621c1f700 1 --2- 172.21.15.62:0/488103890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f461c101950 0x7f461c108200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f461400b0d0 tx=0x7f461400b3e0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.453+0000 7f4621c1f700 1 -- 172.21.15.62:0/488103890 shutdown_connections 2024-05-08T22:36:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.453+0000 7f4621c1f700 1 --2- 172.21.15.62:0/488103890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f461c101950 0x7f461c108200 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.453+0000 7f4621c1f700 1 --2- 172.21.15.62:0/488103890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c0ffdc0 0x7f461c1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.453+0000 7f4621c1f700 1 --2- 172.21.15.62:0/488103890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f461c100fb0 0x7f461c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.453+0000 7f4621c1f700 1 -- 172.21.15.62:0/488103890 >> 172.21.15.62:0/488103890 conn(0x7f461c0fb330 msgr2=0x7f461c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.453+0000 7f4621c1f700 1 -- 172.21.15.62:0/488103890 shutdown_connections 2024-05-08T22:36:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.453+0000 7f4621c1f700 1 -- 172.21.15.62:0/488103890 wait complete. 2024-05-08T22:36:13.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.453+0000 7f4621c1f700 1 Processor -- start 2024-05-08T22:36:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.453+0000 7f4621c1f700 1 -- start start 2024-05-08T22:36:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f4621c1f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f461c0ffdc0 0x7f461c196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f4621c1f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f461c100fb0 0x7f461c197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f4621c1f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c101950 0x7f461c19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f4621c1f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c19e980 con 0x7f461c100fb0 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f4621c1f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c19eac0 con 0x7f461c101950 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f4621c1f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c19ec50 con 0x7f461c0ffdc0 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f461c100fb0 0x7f461c197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f461c100fb0 0x7f461c197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43290/0 (socket says 172.21.15.62:43290) 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461affd700 1 -- 172.21.15.62:0/4070727173 learned_addr learned my addr 172.21.15.62:0/4070727173 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461b7fe700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f461c0ffdc0 0x7f461c196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461bfff700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c101950 0x7f461c19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461affd700 1 -- 172.21.15.62:0/4070727173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f461c0ffdc0 msgr2=0x7f461c196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461affd700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f461c0ffdc0 0x7f461c196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461affd700 1 -- 172.21.15.62:0/4070727173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c101950 msgr2=0x7f461c19e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461affd700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c101950 0x7f461c19e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461affd700 1 -- 172.21.15.62:0/4070727173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4614009d00 con 0x7f461c100fb0 2024-05-08T22:36:13.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461b7fe700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f461c0ffdc0 0x7f461c196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:13.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461bfff700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c101950 0x7f461c19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:13.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.454+0000 7f461affd700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f461c100fb0 0x7f461c197390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c920 tx=0x7f460c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.460+0000 7f4618ff9700 1 -- 172.21.15.62:0/4070727173 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c007b00 con 0x7f461c100fb0 2024-05-08T22:36:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.460+0000 7f4621c1f700 1 -- 172.21.15.62:0/4070727173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c19ef30 con 0x7f461c100fb0 2024-05-08T22:36:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.460+0000 7f4618ff9700 1 -- 172.21.15.62:0/4070727173 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f460c004cf0 con 0x7f461c100fb0 2024-05-08T22:36:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.460+0000 7f4618ff9700 1 -- 172.21.15.62:0/4070727173 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c005520 con 0x7f461c100fb0 2024-05-08T22:36:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.460+0000 7f4621c1f700 1 -- 172.21.15.62:0/4070727173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c19f3b0 con 0x7f461c100fb0 2024-05-08T22:36:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.463+0000 7f4618ff9700 1 -- 172.21.15.62:0/4070727173 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f460c004750 con 0x7f461c100fb0 2024-05-08T22:36:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.463+0000 7f4618ff9700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46080a11b0 0x7f46080a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.463+0000 7f4618ff9700 1 -- 172.21.15.62:0/4070727173 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f460c0cc6f0 con 0x7f461c100fb0 2024-05-08T22:36:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.463+0000 7f461b7fe700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46080a11b0 0x7f46080a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.464+0000 7f461b7fe700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46080a11b0 0x7f46080a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f461c100e10 tx=0x7f460400a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.464+0000 7f462ddd9b80 1 Processor -- start 2024-05-08T22:36:13.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.464+0000 7f462ddd9b80 1 -- start start 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c754f0 0x556e44c75910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c75e50 0x556e44c762d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c76c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44bcc100 con 0x556e44c76810 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44bc9ac0 con 0x556e44c754f0 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44bc53b0 con 0x556e44c75e50 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c76c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c76c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43292/0 (socket says 172.21.15.62:43292) 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461bfff700 1 -- 172.21.15.62:0/2534336835 learned_addr learned my addr 172.21.15.62:0/2534336835 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461affd700 1 --2- 172.21.15.62:0/2534336835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c75e50 0x556e44c762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461b7fe700 1 --2- 172.21.15.62:0/2534336835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c754f0 0x556e44c75910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461bfff700 1 -- 172.21.15.62:0/2534336835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c75e50 msgr2=0x556e44c762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461bfff700 1 --2- 172.21.15.62:0/2534336835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c75e50 0x556e44c762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461bfff700 1 -- 172.21.15.62:0/2534336835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c754f0 msgr2=0x556e44c75910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461bfff700 1 --2- 172.21.15.62:0/2534336835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c754f0 0x556e44c75910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461bfff700 1 -- 172.21.15.62:0/2534336835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e44a46360 con 0x556e44c76810 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461affd700 1 --2- 172.21.15.62:0/2534336835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c75e50 0x556e44c762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f461bfff700 1 --2- 172.21.15.62:0/2534336835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c76c90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4614015000 tx=0x7f461400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36b6408f13e9ce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f4611ffb700 1 -- 172.21.15.62:0/2534336835 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461400e070 con 0x556e44c76810 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f4611ffb700 1 -- 172.21.15.62:0/2534336835 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4614007720 con 0x556e44c76810 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.465+0000 7f4611ffb700 1 -- 172.21.15.62:0/2534336835 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614005440 con 0x556e44c76810 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.466+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2534336835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 msgr2=0x556e44c76c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.466+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/2534336835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c76c90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4614015000 tx=0x7f461400ceb0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.466+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2534336835 shutdown_connections 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.466+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/2534336835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c76c90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.466+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/2534336835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c754f0 0x556e44c75910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.466+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/2534336835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c75e50 0x556e44c762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.466+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2534336835 >> 172.21.15.62:0/2534336835 conn(0x556e44bd31c0 msgr2=0x556e44c6fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.466+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2534336835 shutdown_connections 2024-05-08T22:36:13.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.466+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2534336835 wait complete. 2024-05-08T22:36:13.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f462ddd9b80 1 -- start start 2024-05-08T22:36:13.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f462ddd9b80 1 Processor -- start 2024-05-08T22:36:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c754f0 0x556e44c7ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c75e50 0x556e44c7f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c76810 0x556e44c862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44b61510 con 0x556e44c754f0 2024-05-08T22:36:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44b611b0 con 0x556e44c75e50 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44a48650 con 0x556e44c76810 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f461b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c754f0 0x556e44c7ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f462ddd9b80 10 client.0 _ll_register_callbacks cb 0x556e44b6aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f461b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c754f0 0x556e44c7ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43300/0 (socket says 172.21.15.62:43300) 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f461b7fe700 1 -- 172.21.15.62:0/2683721318 learned_addr learned my addr 172.21.15.62:0/2683721318 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f461affd700 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c75e50 0x556e44c7f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f461bfff700 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c76810 0x556e44c862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f461b7fe700 1 -- 172.21.15.62:0/2683721318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c76810 msgr2=0x556e44c862c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f461b7fe700 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c76810 0x556e44c862c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f461b7fe700 1 -- 172.21.15.62:0/2683721318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c75e50 msgr2=0x556e44c7f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f461b7fe700 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c75e50 0x556e44c7f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f461b7fe700 1 -- 172.21.15.62:0/2683721318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c108b50 con 0x556e44c754f0 2024-05-08T22:36:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.467+0000 7f461bfff700 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c76810 0x556e44c862c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f461b7fe700 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c754f0 0x556e44c7ed40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f460400aa80 tx=0x7f4604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba911ba5af1abc8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f45faffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f45faffd700 1 -- 172.21.15.62:0/2683721318 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604004230 con 0x556e44c754f0 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f45faffd700 1 -- 172.21.15.62:0/2683721318 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4604004390 con 0x556e44c754f0 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2683721318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e44b1e220 con 0x556e44c754f0 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f45faffd700 1 -- 172.21.15.62:0/2683721318 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4604004550 con 0x556e44c754f0 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f462ddd9b80 10 client.15297 fetch_fsmap learned FSMap version 26 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2683721318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556e449f2b10 con 0x556e44c754f0 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f45faffd700 1 -- 172.21.15.62:0/2683721318 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556e44b1e220 con 0x556e44c754f0 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f45faffd700 1 -- 172.21.15.62:0/2683721318 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f460402f070 con 0x556e44c754f0 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f462ddd9b80 10 client.15297 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f462ddd9b80 10 client.15297 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f462ddd9b80 20 client.15297 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x556e44c894c0 0x556e44c8f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f462ddd9b80 4 client.15297 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.468+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2683721318 --> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e44c447c0 con 0x556e44c894c0 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.469+0000 7f461affd700 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x556e44c894c0 0x556e44c8f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.469+0000 7f461affd700 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x556e44c894c0 0x556e44c8f310 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a2a40dd0f6822f6 server_cookie=ed92b7e4408699bb in_seq=0 out_seq=0 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.469+0000 7f45faffd700 10 client.15297 ms_handle_connect on v2:172.21.15.66:6836/1178845415 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f45faffd700 1 -- 172.21.15.62:0/2683721318 <== mds.3 v2:172.21.15.66:6836/1178845415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556e44c447c0 con 0x556e44c894c0 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f45faffd700 10 client.15297 handle_command_reply: tid=0 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 1 client.15297 shutdown 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2683721318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c754f0 msgr2=0x556e44c7ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c754f0 0x556e44c7ed40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f460400aa80 tx=0x7f4604004750 comp rx=0 tx=0).stop 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2683721318 shutdown_connections 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x556e44c894c0 0x556e44c8f310 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c76810 0x556e44c862c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c754f0 0x556e44c7ed40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f45faffd700 0 client.15297 ms_handle_reset on v2:172.21.15.66:6836/1178845415 2024-05-08T22:36:13.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/2683721318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c75e50 0x556e44c7f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2683721318 >> 172.21.15.62:0/2683721318 conn(0x556e44bd31c0 msgr2=0x556e44c742f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2683721318 shutdown_connections 2024-05-08T22:36:13.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 1 -- 172.21.15.62:0/2683721318 wait complete. 2024-05-08T22:36:13.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.470+0000 7f462ddd9b80 20 client.15297 trim_cache size 0 max 16384 2024-05-08T22:36:13.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.488+0000 7f462ddd9b80 1 Processor -- start 2024-05-08T22:36:13.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f462ddd9b80 1 -- start start 2024-05-08T22:36:13.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c754f0 0x556e44c706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c75e50 0x556e44c86280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c7cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44b1e220 con 0x556e44c76810 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44b832b0 con 0x556e44c754f0 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44bbe150 con 0x556e44c75e50 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f461bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c7cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f461bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c7cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43314/0 (socket says 172.21.15.62:43314) 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f461bfff700 1 -- 172.21.15.62:0/1793126076 learned_addr learned my addr 172.21.15.62:0/1793126076 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f461b7fe700 1 --2- 172.21.15.62:0/1793126076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c754f0 0x556e44c706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f461bfff700 1 -- 172.21.15.62:0/1793126076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c75e50 msgr2=0x556e44c86280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f461bfff700 1 --2- 172.21.15.62:0/1793126076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c75e50 0x556e44c86280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f461bfff700 1 -- 172.21.15.62:0/1793126076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c754f0 msgr2=0x556e44c706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f461bfff700 1 --2- 172.21.15.62:0/1793126076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c754f0 0x556e44c706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f461bfff700 1 -- 172.21.15.62:0/1793126076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e44b8b510 con 0x556e44c76810 2024-05-08T22:36:13.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f461b7fe700 1 --2- 172.21.15.62:0/1793126076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c754f0 0x556e44c706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:13.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f461bfff700 1 --2- 172.21.15.62:0/1793126076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c7cd40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x556e44c70cf0 tx=0x7f4614007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c349e77144f60d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f46127fc700 1 -- 172.21.15.62:0/1793126076 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461400e040 con 0x556e44c76810 2024-05-08T22:36:13.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f46127fc700 1 -- 172.21.15.62:0/1793126076 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4614004a50 con 0x556e44c76810 2024-05-08T22:36:13.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.489+0000 7f46127fc700 1 -- 172.21.15.62:0/1793126076 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461401bb80 con 0x556e44c76810 2024-05-08T22:36:13.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.490+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1793126076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 msgr2=0x556e44c7cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.490+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/1793126076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c7cd40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x556e44c70cf0 tx=0x7f4614007b90 comp rx=0 tx=0).stop 2024-05-08T22:36:13.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.490+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1793126076 shutdown_connections 2024-05-08T22:36:13.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.490+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/1793126076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c7cd40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.490+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/1793126076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c754f0 0x556e44c706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.490+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/1793126076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c75e50 0x556e44c86280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.490+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1793126076 >> 172.21.15.62:0/1793126076 conn(0x556e44bd31c0 msgr2=0x556e44c6c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.490+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1793126076 shutdown_connections 2024-05-08T22:36:13.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.490+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1793126076 wait complete. 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.490+0000 7f462ddd9b80 1 -- start start 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.490+0000 7f462ddd9b80 1 Processor -- start 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c754f0 0x556e44c6ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c75e50 0x556e44c6f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f462ddd9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44c85d20 con 0x556e44c76810 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44c85e90 con 0x556e44c75e50 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f462ddd9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44c86050 con 0x556e44c754f0 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f462ddd9b80 10 client.0 _ll_register_callbacks cb 0x556e44b6aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c857e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43330/0 (socket says 172.21.15.62:43330) 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461bfff700 1 -- 172.21.15.62:0/1795816926 learned_addr learned my addr 172.21.15.62:0/1795816926 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461b7fe700 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c754f0 0x556e44c6ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461affd700 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c75e50 0x556e44c6f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461bfff700 1 -- 172.21.15.62:0/1795816926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c754f0 msgr2=0x556e44c6ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461bfff700 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c754f0 0x556e44c6ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461bfff700 1 -- 172.21.15.62:0/1795816926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c75e50 msgr2=0x556e44c6f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461bfff700 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c75e50 0x556e44c6f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461bfff700 1 -- 172.21.15.62:0/1795816926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e44a46360 con 0x556e44c76810 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461affd700 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c75e50 0x556e44c6f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f461bfff700 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c857e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4614007aa0 tx=0x7f4614009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5431234f7e2e28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.491+0000 7f45faffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f45faffd700 1 -- 172.21.15.62:0/1795816926 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461400e070 con 0x556e44c76810 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f45faffd700 1 -- 172.21.15.62:0/1795816926 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f461401bb90 con 0x556e44c76810 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1795816926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4614007720 con 0x556e44c76810 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f45faffd700 1 -- 172.21.15.62:0/1795816926 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f461401bd50 con 0x556e44c76810 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f462ddd9b80 10 client.15303 fetch_fsmap learned FSMap version 26 2024-05-08T22:36:13.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1795816926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556e44a4ac80 con 0x556e44c76810 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f45faffd700 1 -- 172.21.15.62:0/1795816926 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f461403e020 con 0x556e44c76810 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f45faffd700 1 -- 172.21.15.62:0/1795816926 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f461400b100 con 0x556e44c76810 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f462ddd9b80 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f462ddd9b80 10 client.15303 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f462ddd9b80 20 client.15303 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x556e44ce50a0 0x556e44ce1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f462ddd9b80 4 client.15303 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1795816926 --> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556e44bbde70 con 0x556e44ce50a0 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.492+0000 7f461b7fe700 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x556e44ce50a0 0x556e44ce1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f45faffd700 10 client.15303 ms_handle_connect on v2:172.21.15.66:6836/1178845415 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f461b7fe700 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x556e44ce50a0 0x556e44ce1a30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=79c21c8a06b8a307 server_cookie=705c396dac0d5da6 in_seq=0 out_seq=0 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f45faffd700 1 -- 172.21.15.62:0/1795816926 <== mds.3 v2:172.21.15.66:6836/1178845415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556e44bbde70 con 0x556e44ce50a0 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f45faffd700 10 client.15303 handle_command_reply: tid=0 2024-05-08T22:36:13.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 1 client.15303 shutdown 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1795816926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 msgr2=0x556e44c857e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c857e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4614007aa0 tx=0x7f4614009380 comp rx=0 tx=0).stop 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1795816926 shutdown_connections 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e44c76810 0x556e44c857e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.66:6836/1178845415,v1:172.21.15.66:6837/1178845415] conn(0x556e44ce50a0 0x556e44ce1a30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556e44c754f0 0x556e44c6ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 1 --2- 172.21.15.62:0/1795816926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e44c75e50 0x556e44c6f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f45faffd700 0 client.15303 ms_handle_reset on v2:172.21.15.66:6836/1178845415 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1795816926 >> 172.21.15.62:0/1795816926 conn(0x556e44bd31c0 msgr2=0x556e44c7cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1795816926 shutdown_connections 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 1 -- 172.21.15.62:0/1795816926 wait complete. 2024-05-08T22:36:13.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.493+0000 7f462ddd9b80 20 client.15303 trim_cache size 0 max 16384 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 3, 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24503, 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 58.646750371000003, 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 26, 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 81, 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T22:36:13.500 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 76.243441567999994 2024-05-08T22:36:13.501 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:36:13.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 -- 172.21.15.62:0/4070727173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46080a11b0 msgr2=0x7f46080a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46080a11b0 0x7f46080a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f461c100e10 tx=0x7f460400a680 comp rx=0 tx=0).stop 2024-05-08T22:36:13.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 -- 172.21.15.62:0/4070727173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f461c100fb0 msgr2=0x7f461c197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f461c100fb0 0x7f461c197390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c920 tx=0x7f460c00cce0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 -- 172.21.15.62:0/4070727173 shutdown_connections 2024-05-08T22:36:13.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46080a11b0 0x7f46080a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f461c101950 0x7f461c19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f461c0ffdc0 0x7f461c196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 --2- 172.21.15.62:0/4070727173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f461c100fb0 0x7f461c197390 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 -- 172.21.15.62:0/4070727173 >> 172.21.15.62:0/4070727173 conn(0x7f461c0fb330 msgr2=0x7f461c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 -- 172.21.15.62:0/4070727173 shutdown_connections 2024-05-08T22:36:13.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.494+0000 7f4621c1f700 1 -- 172.21.15.62:0/4070727173 wait complete. 2024-05-08T22:36:13.506 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T22:36:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.634+0000 7f5b2334a700 1 -- 172.21.15.62:0/1860138812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b1c101fd0 msgr2=0x7f5b1c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.634+0000 7f5b2334a700 1 --2- 172.21.15.62:0/1860138812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b1c101fd0 0x7f5b1c102450 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009990 tx=0x7f5b10009ca0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.634+0000 7f5b2334a700 1 -- 172.21.15.62:0/1860138812 shutdown_connections 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.634+0000 7f5b2334a700 1 --2- 172.21.15.62:0/1860138812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b1c102990 0x7f5b1c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.634+0000 7f5b2334a700 1 --2- 172.21.15.62:0/1860138812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b1c100e80 0x7f5b1c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.634+0000 7f5b2334a700 1 --2- 172.21.15.62:0/1860138812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b1c101fd0 0x7f5b1c102450 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.634+0000 7f5b2334a700 1 -- 172.21.15.62:0/1860138812 >> 172.21.15.62:0/1860138812 conn(0x7f5b1c0fc3b0 msgr2=0x7f5b1c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.634+0000 7f5b2334a700 1 -- 172.21.15.62:0/1860138812 shutdown_connections 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.634+0000 7f5b2334a700 1 -- 172.21.15.62:0/1860138812 wait complete. 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b2334a700 1 Processor -- start 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b2334a700 1 -- start start 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b2334a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b1c100e80 0x7f5b1c196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b2334a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b1c101fd0 0x7f5b1c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b2334a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b1c102990 0x7f5b1c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b2334a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c19e890 con 0x7f5b1c101fd0 2024-05-08T22:36:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b2334a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c19ea00 con 0x7f5b1c100e80 2024-05-08T22:36:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b2334a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c19ebc0 con 0x7f5b1c102990 2024-05-08T22:36:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b208e5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b1c101fd0 0x7f5b1c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b208e5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b1c101fd0 0x7f5b1c1972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43348/0 (socket says 172.21.15.62:43348) 2024-05-08T22:36:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b208e5700 1 -- 172.21.15.62:0/1942678453 learned_addr learned my addr 172.21.15.62:0/1942678453 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b218e7700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b1c102990 0x7f5b1c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b210e6700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b1c100e80 0x7f5b1c196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b208e5700 1 -- 172.21.15.62:0/1942678453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b1c102990 msgr2=0x7f5b1c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b208e5700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b1c102990 0x7f5b1c19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b208e5700 1 -- 172.21.15.62:0/1942678453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b1c100e80 msgr2=0x7f5b1c196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b208e5700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b1c100e80 0x7f5b1c196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b208e5700 1 -- 172.21.15.62:0/1942678453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b100095f0 con 0x7f5b1c101fd0 2024-05-08T22:36:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b210e6700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b1c100e80 0x7f5b1c196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.635+0000 7f5b208e5700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b1c101fd0 0x7f5b1c1972e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5b10005910 tx=0x7f5b10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.636+0000 7f5b0e7fc700 1 -- 172.21.15.62:0/1942678453 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1001d070 con 0x7f5b1c101fd0 2024-05-08T22:36:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.636+0000 7f5b0e7fc700 1 -- 172.21.15.62:0/1942678453 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b100040e0 con 0x7f5b1c101fd0 2024-05-08T22:36:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.636+0000 7f5b2334a700 1 -- 172.21.15.62:0/1942678453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c19ee40 con 0x7f5b1c101fd0 2024-05-08T22:36:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.636+0000 7f5b0e7fc700 1 -- 172.21.15.62:0/1942678453 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1000f460 con 0x7f5b1c101fd0 2024-05-08T22:36:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.636+0000 7f5b2334a700 1 -- 172.21.15.62:0/1942678453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c19f1b0 con 0x7f5b1c101fd0 2024-05-08T22:36:13.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.638+0000 7f5b0e7fc700 1 -- 172.21.15.62:0/1942678453 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5b10022600 con 0x7f5b1c101fd0 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.638+0000 7f5b0e7fc700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5b080a11b0 0x7f5b080a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.638+0000 7f5b0e7fc700 1 -- 172.21.15.62:0/1942678453 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5b100cdf40 con 0x7f5b1c101fd0 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.638+0000 7f5b210e6700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5b080a11b0 0x7f5b080a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b210e6700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5b080a11b0 0x7f5b080a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005fd0 tx=0x7f5b18005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b2f504b80 1 Processor -- start 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b2f504b80 1 -- start start 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f91b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 0x56276f91c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91c810 0x56276f91cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f872100 con 0x56276f91c810 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f86fac0 con 0x56276f91be50 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f86b3b0 con 0x56276f91b4f0 2024-05-08T22:36:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b218e7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91c810 0x56276f91cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b210e6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f91b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.639+0000 7f5b218e7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91c810 0x56276f91cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43358/0 (socket says 172.21.15.62:43358) 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b218e7700 1 -- 172.21.15.62:0/3875115575 learned_addr learned my addr 172.21.15.62:0/3875115575 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b208e5700 1 --2- 172.21.15.62:0/3875115575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 0x56276f91c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b218e7700 1 -- 172.21.15.62:0/3875115575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 msgr2=0x56276f91b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b218e7700 1 --2- 172.21.15.62:0/3875115575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f91b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b218e7700 1 -- 172.21.15.62:0/3875115575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 msgr2=0x56276f91c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b218e7700 1 --2- 172.21.15.62:0/3875115575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 0x56276f91c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b218e7700 1 -- 172.21.15.62:0/3875115575 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56276f6ec360 con 0x56276f91c810 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b208e5700 1 --2- 172.21.15.62:0/3875115575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 0x56276f91c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b218e7700 1 --2- 172.21.15.62:0/3875115575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91c810 0x56276f91cc90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400a730 tx=0x7f5b140078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36870422da25d659 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b077fe700 1 -- 172.21.15.62:0/3875115575 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14018070 con 0x56276f91c810 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b077fe700 1 -- 172.21.15.62:0/3875115575 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b14010040 con 0x56276f91c810 2024-05-08T22:36:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.640+0000 7f5b077fe700 1 -- 172.21.15.62:0/3875115575 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400f050 con 0x56276f91c810 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.641+0000 7f5b2f504b80 1 -- 172.21.15.62:0/3875115575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91c810 msgr2=0x56276f91cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.641+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/3875115575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91c810 0x56276f91cc90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400a730 tx=0x7f5b140078d0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.641+0000 7f5b2f504b80 1 -- 172.21.15.62:0/3875115575 shutdown_connections 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.641+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/3875115575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91c810 0x56276f91cc90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.641+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/3875115575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f91b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.641+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/3875115575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 0x56276f91c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.641+0000 7f5b2f504b80 1 -- 172.21.15.62:0/3875115575 >> 172.21.15.62:0/3875115575 conn(0x56276f8791c0 msgr2=0x56276f915d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.641+0000 7f5b2f504b80 1 -- 172.21.15.62:0/3875115575 shutdown_connections 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.641+0000 7f5b2f504b80 1 -- 172.21.15.62:0/3875115575 wait complete. 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.641+0000 7f5b2f504b80 1 -- start start 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.641+0000 7f5b2f504b80 1 Processor -- start 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91b4f0 0x56276f924d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 0x56276f925260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91c810 0x56276f92c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f807510 con 0x56276f91b4f0 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f8071b0 con 0x56276f91be50 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f6ee650 con 0x56276f91c810 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b2f504b80 10 client.0 _ll_register_callbacks cb 0x56276f810ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b210e6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91b4f0 0x56276f924d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b210e6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91b4f0 0x56276f924d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43364/0 (socket says 172.21.15.62:43364) 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b210e6700 1 -- 172.21.15.62:0/2843376756 learned_addr learned my addr 172.21.15.62:0/2843376756 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b208e5700 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 0x56276f925260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b218e7700 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91c810 0x56276f92c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b210e6700 1 -- 172.21.15.62:0/2843376756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91c810 msgr2=0x56276f92c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b210e6700 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91c810 0x56276f92c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b210e6700 1 -- 172.21.15.62:0/2843376756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 msgr2=0x56276f925260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b210e6700 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 0x56276f925260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b210e6700 1 -- 172.21.15.62:0/2843376756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1400a410 con 0x56276f91b4f0 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b218e7700 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91c810 0x56276f92c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b208e5700 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 0x56276f925260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b210e6700 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91b4f0 0x56276f924d20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800f9d0 tx=0x7f5b18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6221e897bca05 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.642+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2843376756 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1801d640 con 0x56276f91b4f0 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2843376756 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5b1801d7a0 con 0x56276f91b4f0 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2843376756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56276f7c4220 con 0x56276f91b4f0 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2843376756 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b1801d960 con 0x56276f91b4f0 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b2f504b80 10 client.15315 fetch_fsmap learned FSMap version 26 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2843376756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56276f698b10 con 0x56276f91b4f0 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2843376756 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56276f7c4220 con 0x56276f91b4f0 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2843376756 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5b1802f070 con 0x56276f91b4f0 2024-05-08T22:36:13.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b2f504b80 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b2f504b80 10 client.15315 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b2f504b80 20 client.15315 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x56276f92fee0 0x56276f935200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b2f504b80 4 client.15315 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2843376756 --> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56276f8ea7c0 con 0x56276f92fee0 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.643+0000 7f5b208e5700 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x56276f92fee0 0x56276f935200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.644+0000 7f5b208e5700 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x56276f92fee0 0x56276f935200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5b2bc885c26ce77 server_cookie=c0d6aa0e1715a857 in_seq=0 out_seq=0 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.644+0000 7f5b04ff9700 10 client.15315 ms_handle_connect on v2:172.21.15.79:6836/753714978 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.644+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2843376756 <== mds.2 v2:172.21.15.79:6836/753714978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56276f8ea7c0 con 0x56276f92fee0 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b04ff9700 10 client.15315 handle_command_reply: tid=0 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 1 client.15315 shutdown 2024-05-08T22:36:13.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2843376756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91b4f0 msgr2=0x56276f924d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91b4f0 0x56276f924d20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800f9d0 tx=0x7f5b18004750 comp rx=0 tx=0).stop 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2843376756 shutdown_connections 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x56276f92fee0 0x56276f935200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91c810 0x56276f92c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91b4f0 0x56276f924d20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b04ff9700 0 client.15315 ms_handle_reset on v2:172.21.15.79:6836/753714978 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2843376756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91be50 0x56276f925260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2843376756 >> 172.21.15.62:0/2843376756 conn(0x56276f8791c0 msgr2=0x56276f91a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2843376756 shutdown_connections 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2843376756 wait complete. 2024-05-08T22:36:13.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.645+0000 7f5b2f504b80 20 client.15315 trim_cache size 0 max 16384 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.663+0000 7f5b2f504b80 1 Processor -- start 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.663+0000 7f5b2f504b80 1 -- start start 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f91b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91c810 0x56276f92c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f8292e0 con 0x56276f91be50 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f864180 con 0x56276f91c810 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f80a5a0 con 0x56276f91b4f0 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b208e5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b208e5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43370/0 (socket says 172.21.15.62:43370) 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b208e5700 1 -- 172.21.15.62:0/1326702172 learned_addr learned my addr 172.21.15.62:0/1326702172 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b218e7700 1 --2- 172.21.15.62:0/1326702172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91c810 0x56276f92c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b210e6700 1 --2- 172.21.15.62:0/1326702172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f91b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b208e5700 1 -- 172.21.15.62:0/1326702172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 msgr2=0x56276f91b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b208e5700 1 --2- 172.21.15.62:0/1326702172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f91b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b208e5700 1 -- 172.21.15.62:0/1326702172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91c810 msgr2=0x56276f92c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b208e5700 1 --2- 172.21.15.62:0/1326702172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91c810 0x56276f92c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b208e5700 1 -- 172.21.15.62:0/1326702172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56276f831510 con 0x56276f91be50 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b208e5700 1 --2- 172.21.15.62:0/1326702172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c950 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b100978f0 tx=0x7f5b10009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c27e36c4cea8dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b07fff700 1 -- 172.21.15.62:0/1326702172 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1009b3f0 con 0x56276f91be50 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b07fff700 1 -- 172.21.15.62:0/1326702172 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b100a1040 con 0x56276f91be50 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.664+0000 7f5b07fff700 1 -- 172.21.15.62:0/1326702172 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1009ab40 con 0x56276f91be50 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.665+0000 7f5b2f504b80 1 -- 172.21.15.62:0/1326702172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 msgr2=0x56276f92c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.665+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/1326702172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c950 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b100978f0 tx=0x7f5b10009ca0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.665+0000 7f5b2f504b80 1 -- 172.21.15.62:0/1326702172 shutdown_connections 2024-05-08T22:36:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.665+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/1326702172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91c810 0x56276f92c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.665+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/1326702172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f91b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.665+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/1326702172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c950 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.665+0000 7f5b2f504b80 1 -- 172.21.15.62:0/1326702172 >> 172.21.15.62:0/1326702172 conn(0x56276f8791c0 msgr2=0x56276f9121c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.665+0000 7f5b2f504b80 1 -- 172.21.15.62:0/1326702172 shutdown_connections 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.665+0000 7f5b2f504b80 1 -- 172.21.15.62:0/1326702172 wait complete. 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.665+0000 7f5b2f504b80 1 -- start start 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.665+0000 7f5b2f504b80 1 Processor -- start 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f922070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b2f504b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91c810 0x56276f920720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f920d00 con 0x56276f91be50 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f920e70 con 0x56276f91c810 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b2f504b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276f921030 con 0x56276f91b4f0 2024-05-08T22:36:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b2f504b80 10 client.0 _ll_register_callbacks cb 0x56276f810ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b208e5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b208e5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43376/0 (socket says 172.21.15.62:43376) 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b208e5700 1 -- 172.21.15.62:0/2953445158 learned_addr learned my addr 172.21.15.62:0/2953445158 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b218e7700 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91c810 0x56276f920720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b210e6700 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f922070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b208e5700 1 -- 172.21.15.62:0/2953445158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 msgr2=0x56276f922070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b208e5700 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f922070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b208e5700 1 -- 172.21.15.62:0/2953445158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91c810 msgr2=0x56276f920720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b208e5700 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91c810 0x56276f920720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b208e5700 1 -- 172.21.15.62:0/2953445158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c19f1b0 con 0x56276f91be50 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b208e5700 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5b100a32b0 tx=0x7f5b10009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6a66f435714d56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2953445158 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b100cbc50 con 0x56276f91be50 2024-05-08T22:36:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2953445158 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5b100a37e0 con 0x56276f91be50 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.666+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2953445158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b14010040 con 0x56276f91be50 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2953445158 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b1009b3f0 con 0x56276f91be50 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b2f504b80 10 client.15321 fetch_fsmap learned FSMap version 26 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2953445158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56276f6f0c80 con 0x56276f91be50 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2953445158 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b14010040 con 0x56276f91be50 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2953445158 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5b100971e0 con 0x56276f91be50 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b2f504b80 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b2f504b80 10 client.15321 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b2f504b80 20 client.15321 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x56276f92fee0 0x56276f987b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b2f504b80 4 client.15321 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2953445158 --> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56276f863e70 con 0x56276f92fee0 2024-05-08T22:36:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.667+0000 7f5b210e6700 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x56276f92fee0 0x56276f987b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b04ff9700 10 client.15321 ms_handle_connect on v2:172.21.15.79:6836/753714978 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b210e6700 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x56276f92fee0 0x56276f987b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72bbce8144f7b3de server_cookie=b585a4f35c0d1046 in_seq=0 out_seq=0 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b04ff9700 1 -- 172.21.15.62:0/2953445158 <== mds.2 v2:172.21.15.79:6836/753714978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56276f863e70 con 0x56276f92fee0 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b04ff9700 10 client.15321 handle_command_reply: tid=0 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 1 client.15321 shutdown 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2953445158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 msgr2=0x56276f92c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5b100a32b0 tx=0x7f5b10009520 comp rx=0 tx=0).stop 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2953445158 shutdown_connections 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.79:6836/753714978,v1:172.21.15.79:6837/753714978] conn(0x56276f92fee0 0x56276f987b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56276f91c810 0x56276f920720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56276f91b4f0 0x56276f922070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 1 --2- 172.21.15.62:0/2953445158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56276f91be50 0x56276f92c260 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b04ff9700 0 client.15321 ms_handle_reset on v2:172.21.15.79:6836/753714978 2024-05-08T22:36:13.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2953445158 >> 172.21.15.62:0/2953445158 conn(0x56276f8791c0 msgr2=0x56276f923f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2953445158 shutdown_connections 2024-05-08T22:36:13.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 1 -- 172.21.15.62:0/2953445158 wait complete. 2024-05-08T22:36:13.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.668+0000 7f5b2f504b80 20 client.15321 trim_cache size 0 max 16384 2024-05-08T22:36:13.675 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:36:13.675 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:36:13.675 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-05-08T22:36:13.675 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24505, 2024-05-08T22:36:13.675 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-05-08T22:36:13.675 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-05-08T22:36:13.675 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-05-08T22:36:13.676 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 59.831552498000001, 2024-05-08T22:36:13.676 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 26, 2024-05-08T22:36:13.676 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 81, 2024-05-08T22:36:13.676 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T22:36:13.676 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 78.136557647000004 2024-05-08T22:36:13.676 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 -- 172.21.15.62:0/1942678453 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5b080a11b0 msgr2=0x7f5b080a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5b080a11b0 0x7f5b080a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005fd0 tx=0x7f5b18005e40 comp rx=0 tx=0).stop 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 -- 172.21.15.62:0/1942678453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b1c101fd0 msgr2=0x7f5b1c1972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b1c101fd0 0x7f5b1c1972e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5b10005910 tx=0x7f5b10004730 comp rx=0 tx=0).stop 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 -- 172.21.15.62:0/1942678453 shutdown_connections 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b1c102990 0x7f5b1c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f5b080a11b0 0x7f5b080a3670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b1c100e80 0x7f5b1c196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 --2- 172.21.15.62:0/1942678453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b1c101fd0 0x7f5b1c1972e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 -- 172.21.15.62:0/1942678453 >> 172.21.15.62:0/1942678453 conn(0x7f5b1c0fc3b0 msgr2=0x7f5b1c105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 -- 172.21.15.62:0/1942678453 shutdown_connections 2024-05-08T22:36:13.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.669+0000 7f5b2334a700 1 -- 172.21.15.62:0/1942678453 wait complete. 2024-05-08T22:36:13.681 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-08T22:36:13.705 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:13 smithi062 ceph-mon[42571]: pgmap v255: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 7.3 KiB/s, 1 keys/s, 2 objects/s recovering 2024-05-08T22:36:13.705 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:13 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3033470834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:36:13.706 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:13 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/125675125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:36:13.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.809+0000 7f692e8f0700 1 -- 172.21.15.62:0/3854467420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6928102980 msgr2=0x7f69281092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.809+0000 7f692e8f0700 1 --2- 172.21.15.62:0/3854467420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6928102980 0x7f69281092a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f692000b0a0 tx=0x7f692000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.809+0000 7f692e8f0700 1 -- 172.21.15.62:0/3854467420 shutdown_connections 2024-05-08T22:36:13.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.809+0000 7f692e8f0700 1 --2- 172.21.15.62:0/3854467420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6928102980 0x7f69281092a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.809+0000 7f692e8f0700 1 --2- 172.21.15.62:0/3854467420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6928100e70 0x7f6928101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.809+0000 7f692e8f0700 1 --2- 172.21.15.62:0/3854467420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6928101fc0 0x7f6928102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.809+0000 7f692e8f0700 1 -- 172.21.15.62:0/3854467420 >> 172.21.15.62:0/3854467420 conn(0x7f69280fc400 msgr2=0x7f69280fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.809+0000 7f692e8f0700 1 -- 172.21.15.62:0/3854467420 shutdown_connections 2024-05-08T22:36:13.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.809+0000 7f692e8f0700 1 -- 172.21.15.62:0/3854467420 wait complete. 2024-05-08T22:36:13.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692e8f0700 1 Processor -- start 2024-05-08T22:36:13.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692e8f0700 1 -- start start 2024-05-08T22:36:13.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692e8f0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6928100e70 0x7f6928196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692e8f0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6928101fc0 0x7f69281973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692e8f0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6928102980 0x7f692819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692e8f0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819e9d0 con 0x7f6928102980 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692e8f0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819eb10 con 0x7f6928100e70 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692e8f0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819eca0 con 0x7f6928101fc0 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692ce8d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6928102980 0x7f692819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692ce8d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6928102980 0x7f692819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43390/0 (socket says 172.21.15.62:43390) 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692ce8d700 1 -- 172.21.15.62:0/3836422576 learned_addr learned my addr 172.21.15.62:0/3836422576 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f6927fff700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6928100e70 0x7f6928196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f69277fe700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6928101fc0 0x7f69281973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692ce8d700 1 -- 172.21.15.62:0/3836422576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6928101fc0 msgr2=0x7f69281973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692ce8d700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6928101fc0 0x7f69281973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692ce8d700 1 -- 172.21.15.62:0/3836422576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6928100e70 msgr2=0x7f6928196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692ce8d700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6928100e70 0x7f6928196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.810+0000 7f692ce8d700 1 -- 172.21.15.62:0/3836422576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6920009d00 con 0x7f6928102980 2024-05-08T22:36:13.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.811+0000 7f692ce8d700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6928102980 0x7f692819e400 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6920006070 tx=0x7f692000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.811+0000 7f69257fa700 1 -- 172.21.15.62:0/3836422576 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692000e040 con 0x7f6928102980 2024-05-08T22:36:13.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.811+0000 7f69257fa700 1 -- 172.21.15.62:0/3836422576 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6920004c30 con 0x7f6928102980 2024-05-08T22:36:13.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.811+0000 7f69257fa700 1 -- 172.21.15.62:0/3836422576 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692001b6b0 con 0x7f6928102980 2024-05-08T22:36:13.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.811+0000 7f692e8f0700 1 -- 172.21.15.62:0/3836422576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692819ef20 con 0x7f6928102980 2024-05-08T22:36:13.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.811+0000 7f692e8f0700 1 -- 172.21.15.62:0/3836422576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692819f2e0 con 0x7f6928102980 2024-05-08T22:36:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.814+0000 7f69257fa700 1 -- 172.21.15.62:0/3836422576 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f692001d480 con 0x7f6928102980 2024-05-08T22:36:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.814+0000 7f69257fa700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69140a11b0 0x7f69140a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.814+0000 7f69257fa700 1 -- 172.21.15.62:0/3836422576 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f69200ce2f0 con 0x7f6928102980 2024-05-08T22:36:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f6927fff700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69140a11b0 0x7f69140a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f6927fff700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69140a11b0 0x7f69140a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6918009c60 tx=0x7f6918009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f693aaaab80 1 Processor -- start 2024-05-08T22:36:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f693aaaab80 1 -- start start 2024-05-08T22:36:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce1900 0x560c2dce1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dce26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce2c20 0x560c2dce30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f693aaaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2dc34100 con 0x560c2dce2260 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f693aaaab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2dc31ac0 con 0x560c2dce2c20 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f693aaaab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2dc2d3b0 con 0x560c2dce1900 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f69277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dce26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.815+0000 7f69277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dce26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43394/0 (socket says 172.21.15.62:43394) 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f69277fe700 1 -- 172.21.15.62:0/3460118944 learned_addr learned my addr 172.21.15.62:0/3460118944 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f6927fff700 1 --2- 172.21.15.62:0/3460118944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce1900 0x560c2dce1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f692ce8d700 1 --2- 172.21.15.62:0/3460118944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce2c20 0x560c2dce30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f69277fe700 1 -- 172.21.15.62:0/3460118944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce1900 msgr2=0x560c2dce1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f69277fe700 1 --2- 172.21.15.62:0/3460118944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce1900 0x560c2dce1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f69277fe700 1 -- 172.21.15.62:0/3460118944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce2c20 msgr2=0x560c2dce30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f69277fe700 1 --2- 172.21.15.62:0/3460118944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce2c20 0x560c2dce30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f69277fe700 1 -- 172.21.15.62:0/3460118944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c2daae360 con 0x560c2dce2260 2024-05-08T22:36:13.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f69277fe700 1 --2- 172.21.15.62:0/3460118944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dce26e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6910009d80 tx=0x7f6910011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb03299c8d538b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f691e7fc700 1 -- 172.21.15.62:0/3460118944 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6910004230 con 0x560c2dce2260 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f691e7fc700 1 -- 172.21.15.62:0/3460118944 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6910004390 con 0x560c2dce2260 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.816+0000 7f691e7fc700 1 -- 172.21.15.62:0/3460118944 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69100198c0 con 0x560c2dce2260 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.817+0000 7f693aaaab80 1 -- 172.21.15.62:0/3460118944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 msgr2=0x560c2dce26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.817+0000 7f693aaaab80 1 --2- 172.21.15.62:0/3460118944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dce26e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6910009d80 tx=0x7f6910011720 comp rx=0 tx=0).stop 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.817+0000 7f693aaaab80 1 -- 172.21.15.62:0/3460118944 shutdown_connections 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.817+0000 7f693aaaab80 1 --2- 172.21.15.62:0/3460118944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce2c20 0x560c2dce30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.817+0000 7f693aaaab80 1 --2- 172.21.15.62:0/3460118944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce1900 0x560c2dce1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.817+0000 7f693aaaab80 1 --2- 172.21.15.62:0/3460118944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dce26e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.817+0000 7f693aaaab80 1 -- 172.21.15.62:0/3460118944 >> 172.21.15.62:0/3460118944 conn(0x560c2dc3b1c0 msgr2=0x560c2dcd7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.817+0000 7f693aaaab80 1 -- 172.21.15.62:0/3460118944 shutdown_connections 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.817+0000 7f693aaaab80 1 -- 172.21.15.62:0/3460118944 wait complete. 2024-05-08T22:36:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.817+0000 7f693aaaab80 1 -- start start 2024-05-08T22:36:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f693aaaab80 1 Processor -- start 2024-05-08T22:36:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce1900 0x560c2dceb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dceb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dcf2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f693aaaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2dbc9510 con 0x560c2dce2260 2024-05-08T22:36:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f693aaaab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2dbc91b0 con 0x560c2dce1900 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f693aaaab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2dab0650 con 0x560c2dce2c20 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f69277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dceb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f693aaaab80 10 client.0 _ll_register_callbacks cb 0x560c2dbd2ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f69277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dceb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43398/0 (socket says 172.21.15.62:43398) 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f69277fe700 1 -- 172.21.15.62:0/1008062230 learned_addr learned my addr 172.21.15.62:0/1008062230 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f6927fff700 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce1900 0x560c2dceb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f692ce8d700 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dcf2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f69277fe700 1 -- 172.21.15.62:0/1008062230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 msgr2=0x560c2dcf2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f69277fe700 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dcf2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f69277fe700 1 -- 172.21.15.62:0/1008062230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce1900 msgr2=0x560c2dceb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f69277fe700 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce1900 0x560c2dceb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f69277fe700 1 -- 172.21.15.62:0/1008062230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69100099e0 con 0x560c2dce2260 2024-05-08T22:36:13.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f69277fe700 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dceb6d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6910004360 tx=0x7f69100039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e29231f64defed9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.818+0000 7f69077fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f69077fe700 1 -- 172.21.15.62:0/1008062230 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6910005450 con 0x560c2dce2260 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f69077fe700 1 -- 172.21.15.62:0/1008062230 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6910005ce0 con 0x560c2dce2260 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f693aaaab80 1 -- 172.21.15.62:0/1008062230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c2db86220 con 0x560c2dce2260 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f69077fe700 1 -- 172.21.15.62:0/1008062230 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6910009ed0 con 0x560c2dce2260 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f693aaaab80 10 client.15333 fetch_fsmap learned FSMap version 26 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f693aaaab80 1 -- 172.21.15.62:0/1008062230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560c2da5ab10 con 0x560c2dce2260 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f69077fe700 1 -- 172.21.15.62:0/1008062230 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c2db86220 con 0x560c2dce2260 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f69077fe700 1 -- 172.21.15.62:0/1008062230 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6910021040 con 0x560c2dce2260 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f693aaaab80 10 client.15333 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f693aaaab80 10 client.15333 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f693aaaab80 20 client.15333 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f693aaaab80 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x560c2dcf62c0 0x560c2dcfb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f693aaaab80 4 client.15333 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.819+0000 7f693aaaab80 1 -- 172.21.15.62:0/1008062230 --> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c2dcac7c0 con 0x560c2dcf62c0 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.820+0000 7f6927fff700 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x560c2dcf62c0 0x560c2dcfb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.820+0000 7f69077fe700 10 client.15333 ms_handle_connect on v2:172.21.15.79:6838/3534785027 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.820+0000 7f6927fff700 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x560c2dcf62c0 0x560c2dcfb5c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b22a9236c9232cd8 server_cookie=ab448bae154d272a in_seq=0 out_seq=0 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f69077fe700 1 -- 172.21.15.62:0/1008062230 <== mds.1 v2:172.21.15.79:6838/3534785027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c2dcac7c0 con 0x560c2dcf62c0 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f69077fe700 10 client.15333 handle_command_reply: tid=0 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 1 client.15333 shutdown 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 1 -- 172.21.15.62:0/1008062230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 msgr2=0x560c2dceb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dceb6d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6910004360 tx=0x7f69100039e0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 1 -- 172.21.15.62:0/1008062230 shutdown_connections 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dcf2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce1900 0x560c2dceb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x560c2dcf62c0 0x560c2dcfb5c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 1 --2- 172.21.15.62:0/1008062230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dceb6d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 1 -- 172.21.15.62:0/1008062230 >> 172.21.15.62:0/1008062230 conn(0x560c2dc3b1c0 msgr2=0x560c2dcd7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f69077fe700 0 client.15333 ms_handle_reset on v2:172.21.15.79:6838/3534785027 2024-05-08T22:36:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 1 -- 172.21.15.62:0/1008062230 shutdown_connections 2024-05-08T22:36:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 1 -- 172.21.15.62:0/1008062230 wait complete. 2024-05-08T22:36:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.821+0000 7f693aaaab80 20 client.15333 trim_cache size 0 max 16384 2024-05-08T22:36:13.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:13 smithi066 ceph-mon[46830]: pgmap v255: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 7.3 KiB/s, 1 keys/s, 2 objects/s recovering 2024-05-08T22:36:13.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:13 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3033470834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:36:13.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:13 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/125675125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:36:13.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f693aaaab80 1 Processor -- start 2024-05-08T22:36:13.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f693aaaab80 1 -- start start 2024-05-08T22:36:13.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce1900 0x560c2dcea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce2260 0x560c2dcf26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dceaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f693aaaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2dcac7c0 con 0x560c2dce1900 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f693aaaab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2db86220 con 0x560c2dce2260 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f693aaaab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2dc26150 con 0x560c2dce2c20 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f6927fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce1900 0x560c2dcea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f6927fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce1900 0x560c2dcea6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43410/0 (socket says 172.21.15.62:43410) 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f6927fff700 1 -- 172.21.15.62:0/3919109821 learned_addr learned my addr 172.21.15.62:0/3919109821 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f69277fe700 1 --2- 172.21.15.62:0/3919109821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce2260 0x560c2dcf26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f692ce8d700 1 --2- 172.21.15.62:0/3919109821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dceaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f6927fff700 1 -- 172.21.15.62:0/3919109821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 msgr2=0x560c2dceaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f6927fff700 1 --2- 172.21.15.62:0/3919109821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dceaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f6927fff700 1 -- 172.21.15.62:0/3919109821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce2260 msgr2=0x560c2dcf26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f6927fff700 1 --2- 172.21.15.62:0/3919109821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce2260 0x560c2dcf26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f6927fff700 1 -- 172.21.15.62:0/3919109821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c2dbf3510 con 0x560c2dce1900 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.842+0000 7f69277fe700 1 --2- 172.21.15.62:0/3919109821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce2260 0x560c2dcf26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.843+0000 7f6927fff700 1 --2- 172.21.15.62:0/3919109821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce1900 0x560c2dcea6a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x560c2dcebd20 tx=0x7f6918014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2d5693846c53e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.843+0000 7f691effd700 1 -- 172.21.15.62:0/3919109821 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918022070 con 0x560c2dce1900 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.843+0000 7f691effd700 1 -- 172.21.15.62:0/3919109821 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6918023040 con 0x560c2dce1900 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.843+0000 7f691effd700 1 -- 172.21.15.62:0/3919109821 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180297d0 con 0x560c2dce1900 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.843+0000 7f693aaaab80 1 -- 172.21.15.62:0/3919109821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce1900 msgr2=0x560c2dcea6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.843+0000 7f693aaaab80 1 --2- 172.21.15.62:0/3919109821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce1900 0x560c2dcea6a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x560c2dcebd20 tx=0x7f6918014000 comp rx=0 tx=0).stop 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 -- 172.21.15.62:0/3919109821 shutdown_connections 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 --2- 172.21.15.62:0/3919109821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dceaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 --2- 172.21.15.62:0/3919109821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce1900 0x560c2dcea6a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 --2- 172.21.15.62:0/3919109821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce2260 0x560c2dcf26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 -- 172.21.15.62:0/3919109821 >> 172.21.15.62:0/3919109821 conn(0x560c2dc3b1c0 msgr2=0x560c2dcebe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 -- 172.21.15.62:0/3919109821 shutdown_connections 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 -- 172.21.15.62:0/3919109821 wait complete. 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 -- start start 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 Processor -- start 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce1900 0x560c2dcecc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dced1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dcf2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2dcf2600 con 0x560c2dce2260 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2dcf2770 con 0x560c2dce1900 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2dcf2930 con 0x560c2dce2c20 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f693aaaab80 10 client.0 _ll_register_callbacks cb 0x560c2dbd2ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f69277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dced1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f69277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dced1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43414/0 (socket says 172.21.15.62:43414) 2024-05-08T22:36:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.844+0000 7f69277fe700 1 -- 172.21.15.62:0/682904245 learned_addr learned my addr 172.21.15.62:0/682904245 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f692ce8d700 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dcf2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f6927fff700 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce1900 0x560c2dcecc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f69277fe700 1 -- 172.21.15.62:0/682904245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 msgr2=0x560c2dcf2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f69277fe700 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dcf2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f69277fe700 1 -- 172.21.15.62:0/682904245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce1900 msgr2=0x560c2dcecc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f69277fe700 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce1900 0x560c2dcecc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f69277fe700 1 -- 172.21.15.62:0/682904245 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6918017070 con 0x560c2dce2260 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f69277fe700 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dced1c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6910006010 tx=0x7f6910019800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af94c739e313beb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f69077fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f69077fe700 1 -- 172.21.15.62:0/682904245 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69100198d0 con 0x560c2dce2260 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f69077fe700 1 -- 172.21.15.62:0/682904245 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6910000e40 con 0x560c2dce2260 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f693aaaab80 1 -- 172.21.15.62:0/682904245 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6910004390 con 0x560c2dce2260 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f69077fe700 1 -- 172.21.15.62:0/682904245 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6910003b00 con 0x560c2dce2260 2024-05-08T22:36:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f693aaaab80 10 client.15339 fetch_fsmap learned FSMap version 26 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.845+0000 7f693aaaab80 1 -- 172.21.15.62:0/682904245 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560c2dab2c80 con 0x560c2dce2260 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.846+0000 7f69077fe700 1 -- 172.21.15.62:0/682904245 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6910004390 con 0x560c2dce2260 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.846+0000 7f69077fe700 1 -- 172.21.15.62:0/682904245 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6910021070 con 0x560c2dce2260 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.846+0000 7f693aaaab80 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.846+0000 7f693aaaab80 10 client.15339 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.846+0000 7f693aaaab80 20 client.15339 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.846+0000 7f693aaaab80 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x560c2dd51890 0x560c2dd4e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.846+0000 7f693aaaab80 4 client.15339 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.846+0000 7f693aaaab80 1 -- 172.21.15.62:0/682904245 --> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560c2dc25e70 con 0x560c2dd51890 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.846+0000 7f6927fff700 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x560c2dd51890 0x560c2dd4e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.846+0000 7f6927fff700 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x560c2dd51890 0x560c2dd4e240 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c987422f77facf70 server_cookie=189d2dee0bf06bb2 in_seq=0 out_seq=0 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.846+0000 7f69077fe700 10 client.15339 ms_handle_connect on v2:172.21.15.79:6838/3534785027 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f69077fe700 1 -- 172.21.15.62:0/682904245 <== mds.1 v2:172.21.15.79:6838/3534785027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560c2dc25e70 con 0x560c2dd51890 2024-05-08T22:36:13.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f69077fe700 10 client.15339 handle_command_reply: tid=0 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 1 client.15339 shutdown 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 1 -- 172.21.15.62:0/682904245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 msgr2=0x560c2dced1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dced1c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6910006010 tx=0x7f6910019800 comp rx=0 tx=0).stop 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 1 -- 172.21.15.62:0/682904245 shutdown_connections 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560c2dce2c20 0x560c2dcf2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560c2dce1900 0x560c2dcecc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.79:6838/3534785027,v1:172.21.15.79:6839/3534785027] conn(0x560c2dd51890 0x560c2dd4e240 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 1 --2- 172.21.15.62:0/682904245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c2dce2260 0x560c2dced1c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f69077fe700 0 client.15339 ms_handle_reset on v2:172.21.15.79:6838/3534785027 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 1 -- 172.21.15.62:0/682904245 >> 172.21.15.62:0/682904245 conn(0x560c2dc3b1c0 msgr2=0x560c2dcea650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 1 -- 172.21.15.62:0/682904245 shutdown_connections 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 1 -- 172.21.15.62:0/682904245 wait complete. 2024-05-08T22:36:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f693aaaab80 20 client.15339 trim_cache size 0 max 16384 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24529, 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 61.017909600000003, 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 26, 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 81, 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T22:36:13.854 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 74.973066356999993 2024-05-08T22:36:13.855 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:36:13.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f692e8f0700 1 -- 172.21.15.62:0/3836422576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69140a11b0 msgr2=0x7f69140a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f692e8f0700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69140a11b0 0x7f69140a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6918009c60 tx=0x7f6918009260 comp rx=0 tx=0).stop 2024-05-08T22:36:13.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f692e8f0700 1 -- 172.21.15.62:0/3836422576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6928102980 msgr2=0x7f692819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:13.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.847+0000 7f692e8f0700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6928102980 0x7f692819e400 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6920006070 tx=0x7f692000ba50 comp rx=0 tx=0).stop 2024-05-08T22:36:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.848+0000 7f692e8f0700 1 -- 172.21.15.62:0/3836422576 shutdown_connections 2024-05-08T22:36:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.848+0000 7f692e8f0700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6928102980 0x7f692819e400 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.848+0000 7f692e8f0700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f69140a11b0 0x7f69140a3670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.848+0000 7f692e8f0700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6928100e70 0x7f6928196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.848+0000 7f692e8f0700 1 --2- 172.21.15.62:0/3836422576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6928101fc0 0x7f69281973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.848+0000 7f692e8f0700 1 -- 172.21.15.62:0/3836422576 >> 172.21.15.62:0/3836422576 conn(0x7f69280fc400 msgr2=0x7f6928105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.848+0000 7f692e8f0700 1 -- 172.21.15.62:0/3836422576 shutdown_connections 2024-05-08T22:36:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:13.848+0000 7f692e8f0700 1 -- 172.21.15.62:0/3836422576 wait complete. 2024-05-08T22:36:13.860 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T22:36:13.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:13 smithi079 ceph-mon[46738]: pgmap v255: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 7.3 KiB/s, 1 keys/s, 2 objects/s recovering 2024-05-08T22:36:13.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:13 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3033470834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:36:13.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:13 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/125675125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:36:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.028+0000 7f3a8cda5700 1 -- 172.21.15.62:0/686349874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a88100f90 msgr2=0x7f3a881013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.028+0000 7f3a8cda5700 1 --2- 172.21.15.62:0/686349874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a88100f90 0x7f3a881013f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3a780099c0 tx=0x7f3a78009cd0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.028+0000 7f3a8cda5700 1 -- 172.21.15.62:0/686349874 shutdown_connections 2024-05-08T22:36:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.028+0000 7f3a8cda5700 1 --2- 172.21.15.62:0/686349874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a88101930 0x7f3a881081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.028+0000 7f3a8cda5700 1 --2- 172.21.15.62:0/686349874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a880ffda0 0x7f3a881001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.028+0000 7f3a8cda5700 1 --2- 172.21.15.62:0/686349874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a88100f90 0x7f3a881013f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.028+0000 7f3a8cda5700 1 -- 172.21.15.62:0/686349874 >> 172.21.15.62:0/686349874 conn(0x7f3a880fb2d0 msgr2=0x7f3a880fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.028+0000 7f3a8cda5700 1 -- 172.21.15.62:0/686349874 shutdown_connections 2024-05-08T22:36:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.028+0000 7f3a8cda5700 1 -- 172.21.15.62:0/686349874 wait complete. 2024-05-08T22:36:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a8cda5700 1 Processor -- start 2024-05-08T22:36:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a8cda5700 1 -- start start 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a8cda5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a880ffda0 0x7f3a88196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a8cda5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a88100f90 0x7f3a881972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a8cda5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a88101930 0x7f3a8819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a8cda5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8819e890 con 0x7f3a88101930 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a8cda5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8819ea00 con 0x7f3a880ffda0 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a8cda5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8819ebc0 con 0x7f3a88100f90 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a86d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a88101930 0x7f3a8819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a86d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a88101930 0x7f3a8819e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43440/0 (socket says 172.21.15.62:43440) 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a86d9d700 1 -- 172.21.15.62:0/2466725410 learned_addr learned my addr 172.21.15.62:0/2466725410 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a85d9b700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a88100f90 0x7f3a881972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a8659c700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a880ffda0 0x7f3a88196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a86d9d700 1 -- 172.21.15.62:0/2466725410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a88100f90 msgr2=0x7f3a881972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a86d9d700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a88100f90 0x7f3a881972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a86d9d700 1 -- 172.21.15.62:0/2466725410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a880ffda0 msgr2=0x7f3a88196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a86d9d700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a880ffda0 0x7f3a88196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a86d9d700 1 -- 172.21.15.62:0/2466725410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780095f0 con 0x7f3a88101930 2024-05-08T22:36:14.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.029+0000 7f3a86d9d700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a88101930 0x7f3a8819e2c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000ada0 tx=0x7f3a8000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:14.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.030+0000 7f3a7f7fe700 1 -- 172.21.15.62:0/2466725410 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000f070 con 0x7f3a88101930 2024-05-08T22:36:14.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.030+0000 7f3a7f7fe700 1 -- 172.21.15.62:0/2466725410 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a80007b30 con 0x7f3a88101930 2024-05-08T22:36:14.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.030+0000 7f3a8cda5700 1 -- 172.21.15.62:0/2466725410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8819eea0 con 0x7f3a88101930 2024-05-08T22:36:14.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.030+0000 7f3a7f7fe700 1 -- 172.21.15.62:0/2466725410 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80005440 con 0x7f3a88101930 2024-05-08T22:36:14.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.030+0000 7f3a8cda5700 1 -- 172.21.15.62:0/2466725410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8819f270 con 0x7f3a88101930 2024-05-08T22:36:14.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.032+0000 7f3a7f7fe700 1 -- 172.21.15.62:0/2466725410 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3a80004ad0 con 0x7f3a88101930 2024-05-08T22:36:14.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.032+0000 7f3a7f7fe700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3a740a11b0 0x7f3a740a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.033+0000 7f3a8659c700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3a740a11b0 0x7f3a740a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.033+0000 7f3a7f7fe700 1 -- 172.21.15.62:0/2466725410 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3a800cc170 con 0x7f3a88101930 2024-05-08T22:36:14.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.033+0000 7f3a8659c700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3a740a11b0 0x7f3a740a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3a88198220 tx=0x7f3a7000a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:14.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.033+0000 7f3a98f5fb80 1 Processor -- start 2024-05-08T22:36:14.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a98f5fb80 1 -- start start 2024-05-08T22:36:14.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 0x55a907956910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a9079572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907957810 0x55a907957c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9078ad100 con 0x55a907956e50 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9078aaac0 con 0x55a9079564f0 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9078a63b0 con 0x55a907957810 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a85d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a9079572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a85d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a9079572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43452/0 (socket says 172.21.15.62:43452) 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a85d9b700 1 -- 172.21.15.62:0/2367101291 learned_addr learned my addr 172.21.15.62:0/2367101291 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a8659c700 1 --2- 172.21.15.62:0/2367101291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 0x55a907956910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a85d9b700 1 -- 172.21.15.62:0/2367101291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907957810 msgr2=0x55a907957c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a86d9d700 1 --2- 172.21.15.62:0/2367101291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907957810 0x55a907957c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a85d9b700 1 --2- 172.21.15.62:0/2367101291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907957810 0x55a907957c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a85d9b700 1 -- 172.21.15.62:0/2367101291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 msgr2=0x55a907956910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a85d9b700 1 --2- 172.21.15.62:0/2367101291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 0x55a907956910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a85d9b700 1 -- 172.21.15.62:0/2367101291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a907727360 con 0x55a907956e50 2024-05-08T22:36:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.034+0000 7f3a8659c700 1 --2- 172.21.15.62:0/2367101291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 0x55a907956910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.035+0000 7f3a85d9b700 1 --2- 172.21.15.62:0/2367101291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a9079572d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3a780098e0 tx=0x7f3a78005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef333ce45d0fcc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.035+0000 7f3a7cff9700 1 -- 172.21.15.62:0/2367101291 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a78005530 con 0x55a907956e50 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.035+0000 7f3a7cff9700 1 -- 172.21.15.62:0/2367101291 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a7801e040 con 0x55a907956e50 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.035+0000 7f3a7cff9700 1 -- 172.21.15.62:0/2367101291 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a78003df0 con 0x55a907956e50 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.035+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/2367101291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 msgr2=0x55a9079572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.035+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/2367101291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a9079572d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3a780098e0 tx=0x7f3a78005c00 comp rx=0 tx=0).stop 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.035+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/2367101291 shutdown_connections 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.035+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/2367101291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907957810 0x55a907957c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.035+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/2367101291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 0x55a907956910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.035+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/2367101291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a9079572d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.035+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/2367101291 >> 172.21.15.62:0/2367101291 conn(0x55a9078b41c0 msgr2=0x55a907950d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.036+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/2367101291 shutdown_connections 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.036+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/2367101291 wait complete. 2024-05-08T22:36:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.036+0000 7f3a98f5fb80 1 -- start start 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.036+0000 7f3a98f5fb80 1 Processor -- start 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.036+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9079564f0 0x55a90795fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.036+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a907956e50 0x55a907960290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.036+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907957810 0x55a9079672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.036+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a907842510 con 0x55a907957810 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.036+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9078421b0 con 0x55a907956e50 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.036+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a907729650 con 0x55a9079564f0 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a86d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907957810 0x55a9079672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a98f5fb80 10 client.0 _ll_register_callbacks cb 0x55a90784bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a86d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907957810 0x55a9079672d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43458/0 (socket says 172.21.15.62:43458) 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a86d9d700 1 -- 172.21.15.62:0/211056604 learned_addr learned my addr 172.21.15.62:0/211056604 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a85d9b700 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a907956e50 0x55a907960290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a8659c700 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9079564f0 0x55a90795fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a86d9d700 1 -- 172.21.15.62:0/211056604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9079564f0 msgr2=0x55a90795fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a86d9d700 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9079564f0 0x55a90795fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a86d9d700 1 -- 172.21.15.62:0/211056604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a907956e50 msgr2=0x55a907960290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a86d9d700 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a907956e50 0x55a907960290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a86d9d700 1 -- 172.21.15.62:0/211056604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a88108b30 con 0x55a907957810 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a86d9d700 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907957810 0x55a9079672d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3a80095f70 tx=0x7f3a80098ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358334a61bdee42c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a6dffb700 1 -- 172.21.15.62:0/211056604 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8009d170 con 0x55a907957810 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/211056604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9077ff220 con 0x55a907957810 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a6dffb700 1 -- 172.21.15.62:0/211056604 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3a8009d330 con 0x55a907957810 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a6dffb700 1 -- 172.21.15.62:0/211056604 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a800a18f0 con 0x55a907957810 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a98f5fb80 10 client.15351 fetch_fsmap learned FSMap version 26 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/211056604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a9076d3b10 con 0x55a907957810 2024-05-08T22:36:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.037+0000 7f3a6dffb700 1 -- 172.21.15.62:0/211056604 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a8007c020 con 0x55a907957810 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.038+0000 7f3a6dffb700 1 -- 172.21.15.62:0/211056604 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3a800a1560 con 0x55a907957810 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.038+0000 7f3a98f5fb80 10 client.15351 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.038+0000 7f3a98f5fb80 10 client.15351 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.038+0000 7f3a98f5fb80 20 client.15351 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.038+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55a90796ae20 0x55a907970140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.038+0000 7f3a98f5fb80 4 client.15351 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.038+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/211056604 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9079257c0 con 0x55a90796ae20 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.038+0000 7f3a8659c700 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55a90796ae20 0x55a907970140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.038+0000 7f3a8659c700 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55a90796ae20 0x55a907970140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a55c481c05a12b7 server_cookie=e09e69b1fde45bfe in_seq=0 out_seq=0 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.038+0000 7f3a6dffb700 10 client.15351 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a6dffb700 1 -- 172.21.15.62:0/211056604 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9079257c0 con 0x55a90796ae20 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a6dffb700 10 client.15351 handle_command_reply: tid=0 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 1 client.15351 shutdown 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/211056604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907957810 msgr2=0x55a9079672d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907957810 0x55a9079672d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3a80095f70 tx=0x7f3a80098ee0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/211056604 shutdown_connections 2024-05-08T22:36:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55a90796ae20 0x55a907970140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907957810 0x55a9079672d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a9079564f0 0x55a90795fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/211056604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a907956e50 0x55a907960290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/211056604 >> 172.21.15.62:0/211056604 conn(0x55a9078b41c0 msgr2=0x55a907950550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a6dffb700 0 client.15351 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/211056604 shutdown_connections 2024-05-08T22:36:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/211056604 wait complete. 2024-05-08T22:36:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.039+0000 7f3a98f5fb80 20 client.15351 trim_cache size 0 max 16384 2024-05-08T22:36:14.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.057+0000 7f3a98f5fb80 1 Processor -- start 2024-05-08T22:36:14.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a98f5fb80 1 -- start start 2024-05-08T22:36:14.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9079564f0 0x55a907960390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907956e50 0x55a9079568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a907957810 0x55a907967980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9078642b0 con 0x55a9079564f0 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90789f150 con 0x55a907957810 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9078455a0 con 0x55a907956e50 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a8659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9079564f0 0x55a907960390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a8659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9079564f0 0x55a907960390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43474/0 (socket says 172.21.15.62:43474) 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a8659c700 1 -- 172.21.15.62:0/3100838291 learned_addr learned my addr 172.21.15.62:0/3100838291 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a86d9d700 1 --2- 172.21.15.62:0/3100838291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a907957810 0x55a907967980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a8659c700 1 -- 172.21.15.62:0/3100838291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907956e50 msgr2=0x55a9079568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a85d9b700 1 --2- 172.21.15.62:0/3100838291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907956e50 0x55a9079568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a8659c700 1 --2- 172.21.15.62:0/3100838291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907956e50 0x55a9079568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a8659c700 1 -- 172.21.15.62:0/3100838291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a907957810 msgr2=0x55a907967980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a8659c700 1 --2- 172.21.15.62:0/3100838291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a907957810 0x55a907967980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a8659c700 1 -- 172.21.15.62:0/3100838291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a90786c510 con 0x55a9079564f0 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a86d9d700 1 --2- 172.21.15.62:0/3100838291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a907957810 0x55a907967980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a85d9b700 1 --2- 172.21.15.62:0/3100838291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907956e50 0x55a9079568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a8659c700 1 --2- 172.21.15.62:0/3100838291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9079564f0 0x55a907960390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000a520 tx=0x7f3a70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c49cb63506b10b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.058+0000 7f3a7d7fa700 1 -- 172.21.15.62:0/3100838291 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70024040 con 0x55a9079564f0 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a7d7fa700 1 -- 172.21.15.62:0/3100838291 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a700277f0 con 0x55a9079564f0 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a7d7fa700 1 -- 172.21.15.62:0/3100838291 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70004d60 con 0x55a9079564f0 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3100838291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9079564f0 msgr2=0x55a907960390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/3100838291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9079564f0 0x55a907960390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000a520 tx=0x7f3a70014000 comp rx=0 tx=0).stop 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3100838291 shutdown_connections 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/3100838291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a907957810 0x55a907967980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/3100838291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9079564f0 0x55a907960390 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/3100838291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907956e50 0x55a9079568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3100838291 >> 172.21.15.62:0/3100838291 conn(0x55a9078b41c0 msgr2=0x55a907957bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3100838291 shutdown_connections 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3100838291 wait complete. 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.059+0000 7f3a98f5fb80 1 -- start start 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a98f5fb80 1 Processor -- start 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 0x55a907962500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a907962a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a98f5fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907957810 0x55a9079c1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9079673c0 con 0x55a907956e50 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a907967530 con 0x55a9079564f0 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a98f5fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9079676f0 con 0x55a907957810 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a98f5fb80 10 client.0 _ll_register_callbacks cb 0x55a90784bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a85d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a907962a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a85d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a907962a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43480/0 (socket says 172.21.15.62:43480) 2024-05-08T22:36:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a85d9b700 1 -- 172.21.15.62:0/3080652393 learned_addr learned my addr 172.21.15.62:0/3080652393 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a8659c700 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 0x55a907962500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a85d9b700 1 -- 172.21.15.62:0/3080652393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907957810 msgr2=0x55a9079c1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a85d9b700 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907957810 0x55a9079c1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a85d9b700 1 -- 172.21.15.62:0/3080652393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 msgr2=0x55a907962500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a85d9b700 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 0x55a907962500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a85d9b700 1 -- 172.21.15.62:0/3080652393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7000e900 con 0x55a907956e50 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a8659c700 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 0x55a907962500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a85d9b700 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a907962a40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3a780099c0 tx=0x7f3a7800ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1dc99876c57cd5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.060+0000 7f3a6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a6dffb700 1 -- 172.21.15.62:0/3080652393 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a78003930 con 0x55a907956e50 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a6dffb700 1 -- 172.21.15.62:0/3080652393 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3a78003a90 con 0x55a907956e50 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3080652393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a7801e040 con 0x55a907956e50 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a6dffb700 1 -- 172.21.15.62:0/3080652393 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a78003c50 con 0x55a907956e50 2024-05-08T22:36:14.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a98f5fb80 10 client.15357 fetch_fsmap learned FSMap version 26 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3080652393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a90772bc80 con 0x55a907956e50 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a6dffb700 1 -- 172.21.15.62:0/3080652393 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a7801e040 con 0x55a907956e50 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a6dffb700 1 -- 172.21.15.62:0/3080652393 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3a78004b00 con 0x55a907956e50 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a98f5fb80 10 client.15357 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a98f5fb80 10 client.15357 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a98f5fb80 20 client.15357 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55a9079c6a00 0x55a9079c3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a98f5fb80 4 client.15357 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3080652393 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a90789ee70 con 0x55a9079c6a00 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.061+0000 7f3a8659c700 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55a9079c6a00 0x55a9079c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a6dffb700 10 client.15357 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a8659c700 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55a9079c6a00 0x55a9079c3350 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fd845cb803d0a4b server_cookie=613487a07a97c033 in_seq=0 out_seq=0 2024-05-08T22:36:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a6dffb700 1 -- 172.21.15.62:0/3080652393 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a90789ee70 con 0x55a9079c6a00 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a6dffb700 10 client.15357 handle_command_reply: tid=0 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 1 client.15357 shutdown 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3080652393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 msgr2=0x55a907962a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a907962a40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3a780099c0 tx=0x7f3a7800ba70 comp rx=0 tx=0).stop 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3080652393 shutdown_connections 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55a9079c6a00 0x55a9079c3350 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a907957810 0x55a9079c1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9079564f0 0x55a907962500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a6dffb700 0 client.15357 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 1 --2- 172.21.15.62:0/3080652393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a907956e50 0x55a907962a40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3080652393 >> 172.21.15.62:0/3080652393 conn(0x55a9078b41c0 msgr2=0x55a90795f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3080652393 shutdown_connections 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 1 -- 172.21.15.62:0/3080652393 wait complete. 2024-05-08T22:36:14.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.062+0000 7f3a98f5fb80 20 client.15357 trim_cache size 0 max 16384 2024-05-08T22:36:14.070 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:36:14.070 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "49d88ea8-0d8a-11ef-bc97-c7b262605968", 2024-05-08T22:36:14.070 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-05-08T22:36:14.070 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24533, 2024-05-08T22:36:14.070 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-05-08T22:36:14.070 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-05-08T22:36:14.070 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-05-08T22:36:14.070 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 64.256711421999995, 2024-05-08T22:36:14.070 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 26, 2024-05-08T22:36:14.070 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 81, 2024-05-08T22:36:14.071 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 79, 2024-05-08T22:36:14.071 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 69.997162172000003 2024-05-08T22:36:14.071 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:36:14.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 -- 172.21.15.62:0/2466725410 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3a740a11b0 msgr2=0x7f3a740a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3a740a11b0 0x7f3a740a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3a88198220 tx=0x7f3a7000a5e0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 -- 172.21.15.62:0/2466725410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a88101930 msgr2=0x7f3a8819e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a88101930 0x7f3a8819e2c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000ada0 tx=0x7f3a8000c330 comp rx=0 tx=0).stop 2024-05-08T22:36:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 -- 172.21.15.62:0/2466725410 shutdown_connections 2024-05-08T22:36:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a88101930 0x7f3a8819e2c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3a740a11b0 0x7f3a740a3670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a880ffda0 0x7f3a88196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 --2- 172.21.15.62:0/2466725410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a88100f90 0x7f3a881972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 -- 172.21.15.62:0/2466725410 >> 172.21.15.62:0/2466725410 conn(0x7f3a880fb2d0 msgr2=0x7f3a880fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 -- 172.21.15.62:0/2466725410 shutdown_connections 2024-05-08T22:36:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:14.063+0000 7f3a8cda5700 1 -- 172.21.15.62:0/2466725410 wait complete. 2024-05-08T22:36:14.075 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T22:36:14.087 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T22:36:14.088 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T22:36:14.088 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T22:36:14.088 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T22:36:14.120 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T22:36:14.121 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-08T22:36:14.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:14.121 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-05-08T22:36:14.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:14.139 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-05-08T22:36:14.208 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-05-08T22:36:14.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:36:14.210 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-08T22:36:14.210 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T22:36:14.210 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:14.210 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T22:36:14.478 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:36:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.425+0000 7f426b4fe700 1 -- 172.21.15.62:0/899991527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4264069bd0 msgr2=0x7f426410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.425+0000 7f426b4fe700 1 --2- 172.21.15.62:0/899991527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4264069bd0 0x7f426410de00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f425c00b0d0 tx=0x7f425c00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.425+0000 7f426b4fe700 1 -- 172.21.15.62:0/899991527 shutdown_connections 2024-05-08T22:36:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.425+0000 7f426b4fe700 1 --2- 172.21.15.62:0/899991527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4264069bd0 0x7f426410de00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.425+0000 7f426b4fe700 1 --2- 172.21.15.62:0/899991527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4264106fe0 0x7f42641073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.425+0000 7f426b4fe700 1 --2- 172.21.15.62:0/899991527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4264069210 0x7f4264069690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.425+0000 7f426b4fe700 1 -- 172.21.15.62:0/899991527 >> 172.21.15.62:0/899991527 conn(0x7f4264076e70 msgr2=0x7f4264077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.426+0000 7f426b4fe700 1 -- 172.21.15.62:0/899991527 shutdown_connections 2024-05-08T22:36:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.426+0000 7f426b4fe700 1 -- 172.21.15.62:0/899991527 wait complete. 2024-05-08T22:36:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.426+0000 7f426b4fe700 1 Processor -- start 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.426+0000 7f426b4fe700 1 -- start start 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.426+0000 7f426b4fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4264069210 0x7f42640fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.426+0000 7f426b4fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4264069bd0 0x7f42641004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.426+0000 7f426b4fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4264106fe0 0x7f42641009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.426+0000 7f426b4fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42641a5820 con 0x7f4264069210 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.426+0000 7f426b4fe700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42641a5990 con 0x7f4264106fe0 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.426+0000 7f426b4fe700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42641a5b50 con 0x7f4264069bd0 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f426929a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4264069210 0x7f42640fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f426929a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4264069210 0x7f42640fff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43492/0 (socket says 172.21.15.62:43492) 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f426929a700 1 -- 172.21.15.62:0/473357593 learned_addr learned my addr 172.21.15.62:0/473357593 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f4268a99700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4264069bd0 0x7f42641004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f4269a9b700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4264106fe0 0x7f42641009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f426929a700 1 -- 172.21.15.62:0/473357593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4264069bd0 msgr2=0x7f42641004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f426929a700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4264069bd0 0x7f42641004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f426929a700 1 -- 172.21.15.62:0/473357593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4264106fe0 msgr2=0x7f42641009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f426929a700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4264106fe0 0x7f42641009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f426929a700 1 -- 172.21.15.62:0/473357593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c009d00 con 0x7f4264069210 2024-05-08T22:36:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f426929a700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4264069210 0x7f42640fff60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f426000d950 tx=0x7f426000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f42567fc700 1 -- 172.21.15.62:0/473357593 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426000f890 con 0x7f4264069210 2024-05-08T22:36:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f42567fc700 1 -- 172.21.15.62:0/473357593 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4260010460 con 0x7f4264069210 2024-05-08T22:36:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f426b4fe700 1 -- 172.21.15.62:0/473357593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42641a5e30 con 0x7f4264069210 2024-05-08T22:36:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f42567fc700 1 -- 172.21.15.62:0/473357593 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426000e610 con 0x7f4264069210 2024-05-08T22:36:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.427+0000 7f426b4fe700 1 -- 172.21.15.62:0/473357593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42641a62f0 con 0x7f4264069210 2024-05-08T22:36:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.428+0000 7f426b4fe700 1 -- 172.21.15.62:0/473357593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42641094e0 con 0x7f4264069210 2024-05-08T22:36:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.430+0000 7f42567fc700 1 -- 172.21.15.62:0/473357593 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f42600108d0 con 0x7f4264069210 2024-05-08T22:36:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.430+0000 7f42567fc700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42500a1230 0x7f42500a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.430+0000 7f4268a99700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42500a1230 0x7f42500a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.430+0000 7f42567fc700 1 -- 172.21.15.62:0/473357593 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f42600cc6f0 con 0x7f4264069210 2024-05-08T22:36:15.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.431+0000 7f4268a99700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42500a1230 0x7f42500a36f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4264101690 tx=0x7f42580073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:15.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.433+0000 7f42567fc700 1 -- 172.21.15.62:0/473357593 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4260095d30 con 0x7f4264069210 2024-05-08T22:36:15.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:15 smithi062 ceph-mon[42571]: pgmap v256: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s; 6.4 KiB/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:36:15.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.682+0000 7f426b4fe700 1 -- 172.21.15.62:0/473357593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f42641014e0 con 0x7f4264069210 2024-05-08T22:36:15.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.682+0000 7f42567fc700 1 -- 172.21.15.62:0/473357593 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4260016070 con 0x7f4264069210 2024-05-08T22:36:15.683 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T22:36:15.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 -- 172.21.15.62:0/473357593 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42500a1230 msgr2=0x7f42500a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42500a1230 0x7f42500a36f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4264101690 tx=0x7f42580073d0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 -- 172.21.15.62:0/473357593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4264069210 msgr2=0x7f42640fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:15.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4264069210 0x7f42640fff60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f426000d950 tx=0x7f426000dc60 comp rx=0 tx=0).stop 2024-05-08T22:36:15.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 -- 172.21.15.62:0/473357593 shutdown_connections 2024-05-08T22:36:15.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42500a1230 0x7f42500a36f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4264106fe0 0x7f42641009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4264069210 0x7f42640fff60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 --2- 172.21.15.62:0/473357593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4264069bd0 0x7f42641004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:15.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 -- 172.21.15.62:0/473357593 >> 172.21.15.62:0/473357593 conn(0x7f4264076e70 msgr2=0x7f42640feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:15.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 -- 172.21.15.62:0/473357593 shutdown_connections 2024-05-08T22:36:15.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:15.685+0000 7f426b4fe700 1 -- 172.21.15.62:0/473357593 wait complete. 2024-05-08T22:36:15.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:15 smithi066 ceph-mon[46830]: pgmap v256: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s; 6.4 KiB/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:36:15.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:15 smithi079 ceph-mon[46738]: pgmap v256: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s; 6.4 KiB/s, 1 keys/s, 1 objects/s recovering 2024-05-08T22:36:16.068 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T22:36:16.068 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T22:36:16.069 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-05-08T22:36:16.069 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T22:36:16.069 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:16.069 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T22:36:16.069 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T22:36:16.069 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T22:36:16.069 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:36:16.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:16.069 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-05-08T22:36:16.088 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T22:36:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T22:36:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T22:36:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:36:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-08T22:36:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41770sec preferred_lft 41770sec 2024-05-08T22:36:16.090 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::2bc6:2bcc:607:37e4/64 scope link noprefixroute 2024-05-08T22:36:16.090 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-05-08T22:36:16.090 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T22:36:16.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:16.090 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-05-08T22:36:16.090 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-05-08T22:36:16.090 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-05-08T22:36:16.091 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-05-08T22:36:16.091 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-05-08T22:36:16.091 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T22:36:16.091 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-05-08T22:36:16.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:16.260 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T22:36:16.288 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-05-08T22:36:16.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:16.290 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-05-08T22:36:16.363 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-05-08T22:36:16.364 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T22:36:16.364 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T22:36:16.364 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T22:36:16.364 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T22:36:16.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:16.364 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-05-08T22:36:16.364 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-05-08T22:36:16.365 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T22:36:16.365 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T22:36:16.365 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T22:36:16.365 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-05-08T22:36:16.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:16.622 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-05-08T22:36:16.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:16.641 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-05-08T22:36:16.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:16.698 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-05-08T22:36:16.698 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-05-08T22:36:16.698 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:36:16.698 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T22:36:16.698 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-05-08T22:36:16.778 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:16 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/473357593' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T22:36:16.795 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T22:36:16.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:16.796 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-05-08T22:36:16.796 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-05-08T22:36:16.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T22:36:16.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T22:36:16.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T22:36:16.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T22:36:16.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T22:36:16.796 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-05-08T22:36:16.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:16 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/473357593' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T22:36:16.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:16 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/473357593' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T22:36:17.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:17.040 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-05-08T22:36:17.040 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-05-08T22:36:17.040 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-05-08T22:36:17.040 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T22:36:17.040 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-05-08T22:36:17.140 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:36:17.140 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:17.158 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T22:36:17.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T22:36:17.158 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:17.215 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-08T22:36:17.215 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-08T22:36:17.215 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/511d854d-9d6f-47bd-ba9a-35f51f2d3dc7 2024-05-08T22:36:17.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:17.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/511d854d-9d6f-47bd-ba9a-35f51f2d3dc7 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-08T22:36:17.302 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-08T22:36:17.302 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-08T22:36:17.302 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-05-08T22:36:17.392 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-08T22:36:17.392 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-08T22:36:17.392 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-05-08T22:36:17.392 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.66:3300,172.21.15.79:3300" 2024-05-08T22:36:17.392 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:3300,172.21.15.66:3300,172.21.15.79:3300:/volumes/qa/sv_1/511d854d-9d6f-47bd-ba9a-35f51f2d3dc7 2024-05-08T22:36:17.392 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=49d88ea8-0d8a-11ef-bc97-c7b262605968" will pass to kernel 2024-05-08T22:36:17.395 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T22:36:17.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:17.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:17.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:17.430 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T22:36:17.525 INFO:teuthology.orchestra.run.smithi062.stdout:client25048 2024-05-08T22:36:17.525 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:36:17.525 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:36:17.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:17.544 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T22:36:17.638 INFO:teuthology.orchestra.run.smithi062.stdout:client25048 2024-05-08T22:36:17.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:17.639 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T22:36:17.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:17 smithi062 ceph-mon[42571]: pgmap v257: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 5.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-05-08T22:36:17.692 INFO:teuthology.orchestra.run.smithi062.stdout:client25048 2024-05-08T22:36:17.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:36:17.692 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968.client25048/status') 2024-05-08T22:36:17.792 INFO:teuthology.orchestra.run.smithi062.stdout:instance: client.25048 (3)172.21.15.62:0/1126428027 2024-05-08T22:36:17.792 INFO:teuthology.orchestra.run.smithi062.stdout:blocklisted: false 2024-05-08T22:36:17.793 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2024-05-08T22:36:17.793 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2024-05-08T22:36:17.793 INFO:teuthology.orchestra.run.smithi062.stderr:71 bytes copied, 2.3917e-05 s, 3.0 MB/s 2024-05-08T22:36:17.794 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T22:36:17.805 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T22:36:17.805 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-05-08T22:36:17.805 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-08T22:36:17.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:17 smithi066 ceph-mon[46830]: pgmap v257: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 5.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-05-08T22:36:17.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:17 smithi079 ceph-mon[46738]: pgmap v257: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 5.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-05-08T22:36:17.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.933+0000 7fa3c97df700 1 -- 172.21.15.62:0/109655144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c4101920 msgr2=0x7fa3c41081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.933+0000 7fa3c97df700 1 --2- 172.21.15.62:0/109655144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c4101920 0x7fa3c41081d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b0d0 tx=0x7fa3bc00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.933+0000 7fa3c97df700 1 -- 172.21.15.62:0/109655144 shutdown_connections 2024-05-08T22:36:17.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.933+0000 7fa3c97df700 1 --2- 172.21.15.62:0/109655144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c4101920 0x7fa3c41081d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.933+0000 7fa3c97df700 1 --2- 172.21.15.62:0/109655144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3c40ffd90 0x7fa3c41001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.933+0000 7fa3c97df700 1 --2- 172.21.15.62:0/109655144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3c4100f80 0x7fa3c41013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.933+0000 7fa3c97df700 1 -- 172.21.15.62:0/109655144 >> 172.21.15.62:0/109655144 conn(0x7fa3c40fb300 msgr2=0x7fa3c40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:17.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.933+0000 7fa3c97df700 1 -- 172.21.15.62:0/109655144 shutdown_connections 2024-05-08T22:36:17.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.933+0000 7fa3c97df700 1 -- 172.21.15.62:0/109655144 wait complete. 2024-05-08T22:36:17.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.934+0000 7fa3c97df700 1 Processor -- start 2024-05-08T22:36:17.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.934+0000 7fa3c97df700 1 -- start start 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.934+0000 7fa3c97df700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3c40ffd90 0x7fa3c4195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.934+0000 7fa3c97df700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c4100f80 0x7fa3c4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.934+0000 7fa3c97df700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3c4101920 0x7fa3c419d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.934+0000 7fa3c97df700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c419d930 con 0x7fa3c4100f80 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.934+0000 7fa3c97df700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c419da70 con 0x7fa3c4101920 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.934+0000 7fa3c97df700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c419dc00 con 0x7fa3c40ffd90 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c4100f80 0x7fa3c4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c4100f80 0x7fa3c4196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35388/0 (socket says 172.21.15.62:35388) 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c27fc700 1 -- 172.21.15.62:0/3461775568 learned_addr learned my addr 172.21.15.62:0/3461775568 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c37fe700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3c4101920 0x7fa3c419d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c2ffd700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3c40ffd90 0x7fa3c4195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c27fc700 1 -- 172.21.15.62:0/3461775568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3c40ffd90 msgr2=0x7fa3c4195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c27fc700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3c40ffd90 0x7fa3c4195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c27fc700 1 -- 172.21.15.62:0/3461775568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3c4101920 msgr2=0x7fa3c419d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c27fc700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3c4101920 0x7fa3c419d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c27fc700 1 -- 172.21.15.62:0/3461775568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc009d00 con 0x7fa3c4100f80 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c2ffd700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3c40ffd90 0x7fa3c4195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c27fc700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c4100f80 0x7fa3c4196340 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400e9a0 tx=0x7fa3b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3bbfff700 1 -- 172.21.15.62:0/3461775568 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4009790 con 0x7fa3c4100f80 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3bbfff700 1 -- 172.21.15.62:0/3461775568 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3b400cbd0 con 0x7fa3c4100f80 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c97df700 1 -- 172.21.15.62:0/3461775568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c419dee0 con 0x7fa3c4100f80 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3bbfff700 1 -- 172.21.15.62:0/3461775568 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4018680 con 0x7fa3c4100f80 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.935+0000 7fa3c97df700 1 -- 172.21.15.62:0/3461775568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c419e3b0 con 0x7fa3c4100f80 2024-05-08T22:36:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.936+0000 7fa3c97df700 1 -- 172.21.15.62:0/3461775568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c4107140 con 0x7fa3c4100f80 2024-05-08T22:36:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.937+0000 7fa3bbfff700 1 -- 172.21.15.62:0/3461775568 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa3b400cd40 con 0x7fa3c4100f80 2024-05-08T22:36:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.938+0000 7fa3bbfff700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3b00a1220 0x7fa3b00a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.938+0000 7fa3bbfff700 1 -- 172.21.15.62:0/3461775568 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa3b4014070 con 0x7fa3c4100f80 2024-05-08T22:36:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.938+0000 7fa3c2ffd700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3b00a1220 0x7fa3b00a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:17.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.938+0000 7fa3c2ffd700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3b00a1220 0x7fa3b00a36e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4100de0 tx=0x7fa3ac009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:17.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:17.942+0000 7fa3bbfff700 1 -- 172.21.15.62:0/3461775568 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa3b40964a0 con 0x7fa3c4100f80 2024-05-08T22:36:18.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.199+0000 7fa3c97df700 1 -- 172.21.15.62:0/3461775568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7fa3c404ec60 con 0x7fa3c4100f80 2024-05-08T22:36:18.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.477+0000 7fa3bbfff700 1 -- 172.21.15.62:0/3461775568 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) v1 ==== 117+0+0 (secure 0 0 0) 0x7fa3b40260b0 con 0x7fa3c4100f80 2024-05-08T22:36:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 -- 172.21.15.62:0/3461775568 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3b00a1220 msgr2=0x7fa3b00a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3b00a1220 0x7fa3b00a36e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4100de0 tx=0x7fa3ac009670 comp rx=0 tx=0).stop 2024-05-08T22:36:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 -- 172.21.15.62:0/3461775568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c4100f80 msgr2=0x7fa3c4196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c4100f80 0x7fa3c4196340 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400e9a0 tx=0x7fa3b400ecb0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 -- 172.21.15.62:0/3461775568 shutdown_connections 2024-05-08T22:36:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fa3b00a1220 0x7fa3b00a36e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3c4101920 0x7fa3c419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3c40ffd90 0x7fa3c4195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 --2- 172.21.15.62:0/3461775568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c4100f80 0x7fa3c4196340 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 -- 172.21.15.62:0/3461775568 >> 172.21.15.62:0/3461775568 conn(0x7fa3c40fb300 msgr2=0x7fa3c4104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:18.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 -- 172.21.15.62:0/3461775568 shutdown_connections 2024-05-08T22:36:18.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:36:18.480+0000 7fa3c97df700 1 -- 172.21.15.62:0/3461775568 wait complete. 2024-05-08T22:36:18.493 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-08T22:36:18.506 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-08T22:36:18.517 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-05-08T22:36:18.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-08T22:36:18.520 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T22:36:18.532 INFO:tasks.workunit:Pulling workunits from ref d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:36:18.533 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T22:36:18.533 INFO:tasks.workunit:timeout=3h 2024-05-08T22:36:18.533 INFO:tasks.workunit:cleanup=True 2024-05-08T22:36:18.534 DEBUG:teuthology.orchestra.run.smithi062:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:18.555 INFO:teuthology.orchestra.run.smithi062.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:18.555 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-08T22:36:18.555 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2024-05-08T22:36:18.555 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T22:36:18.555 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:cephfs_t:s0 2024-05-08T22:36:18.555 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-05-08 22:35:16.522780800 +0000 2024-05-08T22:36:18.555 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-05-08 22:35:16.522780800 +0000 2024-05-08T22:36:18.555 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-05-08 22:36:17.424129755 +0000 2024-05-08T22:36:18.555 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-05-08 22:35:16.522780800 +0000 2024-05-08T22:36:18.555 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-08T22:36:18.556 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-08T22:36:18.636 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:36:18.695 INFO:tasks.workunit.client.0.smithi062.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T22:36:18.765 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:18 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3461775568' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T22:36:18.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:18 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3461775568' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T22:36:18.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:18 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3461775568' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T22:36:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:19 smithi066 ceph-mon[46830]: pgmap v258: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 4.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-05-08T22:36:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:19 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3461775568' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T22:36:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:19 smithi066 ceph-mon[46830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:36:19.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:19 smithi079 ceph-mon[46738]: pgmap v258: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 4.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-05-08T22:36:19.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:19 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3461775568' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T22:36:19.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:19 smithi079 ceph-mon[46738]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:36:19.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:19 smithi062 ceph-mon[42571]: pgmap v258: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 4.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-05-08T22:36:19.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:19 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3461775568' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T22:36:19.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:19 smithi062 ceph-mon[42571]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:36:21.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:21 smithi066 ceph-mon[46830]: pgmap v259: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T22:36:21.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:21 smithi079 ceph-mon[46738]: pgmap v259: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T22:36:21.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:21 smithi062 ceph-mon[42571]: pgmap v259: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T22:36:23.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:23 smithi066 ceph-mon[46830]: pgmap v260: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-05-08T22:36:23.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:23 smithi079 ceph-mon[46738]: pgmap v260: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-05-08T22:36:23.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:23 smithi062 ceph-mon[42571]: pgmap v260: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-05-08T22:36:25.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:25 smithi066 ceph-mon[46830]: pgmap v261: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-08T22:36:25.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:25 smithi066 ceph-mon[46830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:36:25.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:25 smithi079 ceph-mon[46738]: pgmap v261: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-08T22:36:25.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:25 smithi079 ceph-mon[46738]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:36:25.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:25 smithi062 ceph-mon[42571]: pgmap v261: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-08T22:36:25.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:25 smithi062 ceph-mon[42571]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:36:27.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:27 smithi066 ceph-mon[46830]: pgmap v262: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-08T22:36:27.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:27 smithi079 ceph-mon[46738]: pgmap v262: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-08T22:36:27.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:27 smithi062 ceph-mon[42571]: pgmap v262: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-08T22:36:29.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:29 smithi066 ceph-mon[46830]: pgmap v263: 97 pgs: 97 active+clean; 662 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-05-08T22:36:29.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:29 smithi079 ceph-mon[46738]: pgmap v263: 97 pgs: 97 active+clean; 662 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-05-08T22:36:29.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:29 smithi062 ceph-mon[42571]: pgmap v263: 97 pgs: 97 active+clean; 662 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-05-08T22:36:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.692+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/502657056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6d4101930 msgr2=0x7ff6d41081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.692+0000 7ff6dc2f7700 1 --2- 172.21.15.62:0/502657056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6d4101930 0x7ff6d41081e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00b0d0 tx=0x7ff6cc00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.692+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/502657056 shutdown_connections 2024-05-08T22:36:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.692+0000 7ff6dc2f7700 1 --2- 172.21.15.62:0/502657056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6d4101930 0x7ff6d41081e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.692+0000 7ff6dc2f7700 1 --2- 172.21.15.62:0/502657056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d40ffda0 0x7ff6d41001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.692+0000 7ff6dc2f7700 1 --2- 172.21.15.62:0/502657056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6d4100f90 0x7ff6d41013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.692+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/502657056 >> 172.21.15.62:0/502657056 conn(0x7ff6d40fb330 msgr2=0x7ff6d40fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.693+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/502657056 shutdown_connections 2024-05-08T22:36:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.693+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/502657056 wait complete. 2024-05-08T22:36:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.693+0000 7ff6dc2f7700 1 Processor -- start 2024-05-08T22:36:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.693+0000 7ff6dc2f7700 1 -- start start 2024-05-08T22:36:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.693+0000 7ff6dc2f7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6d40ffda0 0x7ff6d4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.693+0000 7ff6dc2f7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d4100f90 0x7ff6d41996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.693+0000 7ff6dc2f7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6d4101930 0x7ff6d41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.693+0000 7ff6dc2f7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41a0c90 con 0x7ff6d4101930 2024-05-08T22:36:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.693+0000 7ff6dc2f7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41a0dd0 con 0x7ff6d40ffda0 2024-05-08T22:36:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.693+0000 7ff6dc2f7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41a0f60 con 0x7ff6d4100f90 2024-05-08T22:36:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6da894700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6d4101930 0x7ff6d41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6da894700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6d4101930 0x7ff6d41a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60902/0 (socket says 172.21.15.62:60902) 2024-05-08T22:36:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6da894700 1 -- 172.21.15.62:0/2258724126 learned_addr learned my addr 172.21.15.62:0/2258724126 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6da093700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6d40ffda0 0x7ff6d4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6d9892700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d4100f90 0x7ff6d41996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6da894700 1 -- 172.21.15.62:0/2258724126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d4100f90 msgr2=0x7ff6d41996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6da894700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d4100f90 0x7ff6d41996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6da894700 1 -- 172.21.15.62:0/2258724126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6d40ffda0 msgr2=0x7ff6d4199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6da894700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6d40ffda0 0x7ff6d4199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6da894700 1 -- 172.21.15.62:0/2258724126 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc009d00 con 0x7ff6d4101930 2024-05-08T22:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6da093700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6d40ffda0 0x7ff6d4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6d9892700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d4100f90 0x7ff6d41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6da894700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6d4101930 0x7ff6d41a06c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc007b60 tx=0x7ff6cc008e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6c77fe700 1 -- 172.21.15.62:0/2258724126 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc00e040 con 0x7ff6d4101930 2024-05-08T22:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6c77fe700 1 -- 172.21.15.62:0/2258724126 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6cc00be30 con 0x7ff6d4101930 2024-05-08T22:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/2258724126 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d41a11e0 con 0x7ff6d4101930 2024-05-08T22:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6c77fe700 1 -- 172.21.15.62:0/2258724126 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc01bb90 con 0x7ff6d4101930 2024-05-08T22:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.694+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/2258724126 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d41a1660 con 0x7ff6d4101930 2024-05-08T22:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.697+0000 7ff6c77fe700 1 -- 172.21.15.62:0/2258724126 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff6cc01d670 con 0x7ff6d4101930 2024-05-08T22:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.697+0000 7ff6c77fe700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff6c00a11a0 0x7ff6c00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.697+0000 7ff6c77fe700 1 -- 172.21.15.62:0/2258724126 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff6cc0cdf70 con 0x7ff6d4101930 2024-05-08T22:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.697+0000 7ff6da093700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff6c00a11a0 0x7ff6c00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.698+0000 7ff6da093700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff6c00a11a0 0x7ff6c00a3660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4100df0 tx=0x7ff6c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.698+0000 7ff6e84b1b80 1 Processor -- start 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.698+0000 7ff6e84b1b80 1 -- start start 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.698+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a5a80 0x55def81a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.698+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81a7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def80fd0f0 con 0x55def81a72d0 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def80faab0 con 0x55def81a5a80 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def80f63a0 con 0x55def81a63e0 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6da894700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6da894700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81a7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60910/0 (socket says 172.21.15.62:60910) 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6da894700 1 -- 172.21.15.62:0/3974242466 learned_addr learned my addr 172.21.15.62:0/3974242466 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6d9892700 1 --2- 172.21.15.62:0/3974242466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6da093700 1 --2- 172.21.15.62:0/3974242466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a5a80 0x55def81a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6da894700 1 -- 172.21.15.62:0/3974242466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 msgr2=0x55def81a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6da894700 1 --2- 172.21.15.62:0/3974242466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6da894700 1 -- 172.21.15.62:0/3974242466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a5a80 msgr2=0x55def81a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6da894700 1 --2- 172.21.15.62:0/3974242466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a5a80 0x55def81a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6da894700 1 -- 172.21.15.62:0/3974242466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55def7f77350 con 0x55def81a72d0 2024-05-08T22:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6da894700 1 --2- 172.21.15.62:0/3974242466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81a7750 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0a31c0 tx=0x7ff6cc00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78089571172a5b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.699+0000 7ff6c4da4700 1 -- 172.21.15.62:0/3974242466 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc09ba00 con 0x55def81a72d0 2024-05-08T22:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.700+0000 7ff6c4da4700 1 -- 172.21.15.62:0/3974242466 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6cc0b0070 con 0x55def81a72d0 2024-05-08T22:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.700+0000 7ff6c4da4700 1 -- 172.21.15.62:0/3974242466 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc02d4f0 con 0x55def81a72d0 2024-05-08T22:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.700+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/3974242466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 msgr2=0x55def81a7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.700+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/3974242466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81a7750 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0a31c0 tx=0x7ff6cc00b3e0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.700+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/3974242466 shutdown_connections 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.700+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/3974242466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81a7750 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.700+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/3974242466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a5a80 0x55def81a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.700+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/3974242466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.700+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/3974242466 >> 172.21.15.62:0/3974242466 conn(0x55def81041b0 msgr2=0x55def81a2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.700+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/3974242466 shutdown_connections 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.700+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/3974242466 wait complete. 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6e84b1b80 1 -- start start 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6e84b1b80 1 Processor -- start 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a72d0 0x55def81b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def8092500 con 0x55def81a5a80 2024-05-08T22:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def80921a0 con 0x55def81a72d0 2024-05-08T22:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def7f79640 con 0x55def81a63e0 2024-05-08T22:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6e84b1b80 10 client.0 _ll_register_callbacks cb 0x55def809bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6da093700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6da093700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60916/0 (socket says 172.21.15.62:60916) 2024-05-08T22:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6da093700 1 -- 172.21.15.62:0/1826678981 learned_addr learned my addr 172.21.15.62:0/1826678981 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6d9892700 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6da093700 1 -- 172.21.15.62:0/1826678981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 msgr2=0x55def81afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.701+0000 7ff6da093700 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6da093700 1 -- 172.21.15.62:0/1826678981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a72d0 msgr2=0x55def81b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6da093700 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a72d0 0x55def81b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6da093700 1 -- 172.21.15.62:0/1826678981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d41a1660 con 0x55def81a5a80 2024-05-08T22:36:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6d9892700 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6da093700 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81af6c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8003aa0 tx=0x7ff6c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed890699c821910 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1826678981 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c801fcd0 con 0x55def81a5a80 2024-05-08T22:36:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1826678981 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff6c801fe30 con 0x55def81a5a80 2024-05-08T22:36:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1826678981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55def804f210 con 0x55def81a5a80 2024-05-08T22:36:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1826678981 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6c8028440 con 0x55def81a5a80 2024-05-08T22:36:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6e84b1b80 10 client.15381 fetch_fsmap learned FSMap version 28 2024-05-08T22:36:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1826678981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55def7f23b00 con 0x55def81a5a80 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1826678981 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55def804f210 con 0x55def81a5a80 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.702+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1826678981 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff6c8032620 con 0x55def81a5a80 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.703+0000 7ff6e84b1b80 10 client.15381 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.703+0000 7ff6e84b1b80 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.703+0000 7ff6e84b1b80 20 client.15381 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.703+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55def81bb5d0 0x55def81bd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.703+0000 7ff6e84b1b80 4 client.15381 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.703+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1826678981 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55def81757b0 con 0x55def81bb5d0 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.703+0000 7ff6d9892700 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55def81bb5d0 0x55def81bd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.703+0000 7ff6d9892700 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55def81bb5d0 0x55def81bd9f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e493caf1b3b7514 server_cookie=807e9e046f780880 in_seq=0 out_seq=0 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.703+0000 7ff6bdffb700 10 client.15381 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1826678981 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55def81757b0 con 0x55def81bb5d0 2024-05-08T22:36:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6bdffb700 10 client.15381 handle_command_reply: tid=0 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 1 client.15381 shutdown 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1826678981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 msgr2=0x55def81af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81af6c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8003aa0 tx=0x7ff6c80047c0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1826678981 shutdown_connections 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55def81bb5d0 0x55def81bd9f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a72d0 0x55def81b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81af6c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6bdffb700 0 client.15381 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1826678981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1826678981 >> 172.21.15.62:0/1826678981 conn(0x55def81041b0 msgr2=0x55def81a0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1826678981 shutdown_connections 2024-05-08T22:36:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1826678981 wait complete. 2024-05-08T22:36:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.704+0000 7ff6e84b1b80 20 client.15381 trim_cache size 0 max 16384 2024-05-08T22:36:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.719+0000 7ff6e84b1b80 1 Processor -- start 2024-05-08T22:36:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.719+0000 7ff6e84b1b80 1 -- start start 2024-05-08T22:36:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.719+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81bd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.719+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.719+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a72d0 0x55def81b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.719+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def81ae370 con 0x55def81a5a80 2024-05-08T22:36:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.719+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def81a1930 con 0x55def81a72d0 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.719+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def81b7a20 con 0x55def81a63e0 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.719+0000 7ff6da093700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81bd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.719+0000 7ff6da093700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81bd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60928/0 (socket says 172.21.15.62:60928) 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6da093700 1 -- 172.21.15.62:0/1494998066 learned_addr learned my addr 172.21.15.62:0/1494998066 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6da894700 1 --2- 172.21.15.62:0/1494998066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a72d0 0x55def81b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6d9892700 1 --2- 172.21.15.62:0/1494998066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6da093700 1 -- 172.21.15.62:0/1494998066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 msgr2=0x55def81a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6da093700 1 --2- 172.21.15.62:0/1494998066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6da093700 1 -- 172.21.15.62:0/1494998066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a72d0 msgr2=0x55def81b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6da093700 1 --2- 172.21.15.62:0/1494998066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a72d0 0x55def81b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6da093700 1 -- 172.21.15.62:0/1494998066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55def80bc500 con 0x55def81a5a80 2024-05-08T22:36:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6da093700 1 --2- 172.21.15.62:0/1494998066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81bd9f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80097c0 tx=0x7ff6c8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486d3cc6d5ae75c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6c55a5700 1 -- 172.21.15.62:0/1494998066 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8038070 con 0x55def81a5a80 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6c55a5700 1 -- 172.21.15.62:0/1494998066 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c80040f0 con 0x55def81a5a80 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.720+0000 7ff6c55a5700 1 -- 172.21.15.62:0/1494998066 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8004a90 con 0x55def81a5a80 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1494998066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 msgr2=0x55def81bd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1494998066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81bd9f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80097c0 tx=0x7ff6c8003f10 comp rx=0 tx=0).stop 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1494998066 shutdown_connections 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1494998066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a72d0 0x55def81b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1494998066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a5a80 0x55def81bd9f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1494998066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a63e0 0x55def81a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1494998066 >> 172.21.15.62:0/1494998066 conn(0x55def81041b0 msgr2=0x55def81ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1494998066 shutdown_connections 2024-05-08T22:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1494998066 wait complete. 2024-05-08T22:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 -- start start 2024-05-08T22:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 Processor -- start 2024-05-08T22:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a5a80 0x55def81b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a63e0 0x55def81b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81b5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def81b5bc0 con 0x55def81a72d0 2024-05-08T22:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def81b5d30 con 0x55def81a63e0 2024-05-08T22:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def81b5ef0 con 0x55def81a5a80 2024-05-08T22:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.721+0000 7ff6e84b1b80 10 client.0 _ll_register_callbacks cb 0x55def809bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6da894700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6da894700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81b5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60942/0 (socket says 172.21.15.62:60942) 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6da894700 1 -- 172.21.15.62:0/1781704712 learned_addr learned my addr 172.21.15.62:0/1781704712 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6d9892700 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a63e0 0x55def81b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6da093700 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a5a80 0x55def81b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6da894700 1 -- 172.21.15.62:0/1781704712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a5a80 msgr2=0x55def81b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6da894700 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a5a80 0x55def81b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6da894700 1 -- 172.21.15.62:0/1781704712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a63e0 msgr2=0x55def81b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6da894700 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a63e0 0x55def81b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6da894700 1 -- 172.21.15.62:0/1781704712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55def7f23b00 con 0x55def81a72d0 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6da894700 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81b5680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0cb9b0 tx=0x7ff6cc0074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826d6e31b3078023 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1781704712 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc09b300 con 0x55def81a72d0 2024-05-08T22:36:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1781704712 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff6cc09ba00 con 0x55def81a72d0 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1781704712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6cc0b0070 con 0x55def81a72d0 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1781704712 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6cc09f130 con 0x55def81a72d0 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6e84b1b80 10 client.15387 fetch_fsmap learned FSMap version 28 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.722+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1781704712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55def7f7bc70 con 0x55def81a72d0 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.723+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1781704712 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6cc0b0070 con 0x55def81a72d0 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.723+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1781704712 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff6cc09b560 con 0x55def81a72d0 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.723+0000 7ff6e84b1b80 10 client.15387 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.723+0000 7ff6e84b1b80 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.723+0000 7ff6e84b1b80 20 client.15387 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.723+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55def8211db0 0x55def82141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.723+0000 7ff6e84b1b80 4 client.15387 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.723+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1781704712 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55def80eee60 con 0x55def8211db0 2024-05-08T22:36:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.723+0000 7ff6da093700 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55def8211db0 0x55def82141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.723+0000 7ff6bdffb700 10 client.15387 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.723+0000 7ff6da093700 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55def8211db0 0x55def82141a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5a8ffd39dc688e0 server_cookie=25f4d76ac3026e84 in_seq=0 out_seq=0 2024-05-08T22:36:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6bdffb700 1 -- 172.21.15.62:0/1781704712 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55def80eee60 con 0x55def8211db0 2024-05-08T22:36:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6bdffb700 10 client.15387 handle_command_reply: tid=0 2024-05-08T22:36:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 1 client.15387 shutdown 2024-05-08T22:36:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1781704712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 msgr2=0x55def81b5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81b5680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0cb9b0 tx=0x7ff6cc0074e0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1781704712 shutdown_connections 2024-05-08T22:36:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55def8211db0 0x55def82141a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def81a72d0 0x55def81b5680 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6bdffb700 0 client.15387 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55def81a5a80 0x55def81b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 1 --2- 172.21.15.62:0/1781704712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55def81a63e0 0x55def81b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1781704712 >> 172.21.15.62:0/1781704712 conn(0x55def81041b0 msgr2=0x55def81ad830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1781704712 shutdown_connections 2024-05-08T22:36:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 1 -- 172.21.15.62:0/1781704712 wait complete. 2024-05-08T22:36:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.735+0000 7ff6e84b1b80 20 client.15387 trim_cache size 0 max 16384 2024-05-08T22:36:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-05-08T22:36:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-05-08T22:36:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-05-08T22:36:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-05-08T22:36:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/2258724126 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff6c00a11a0 msgr2=0x7ff6c00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff6c00a11a0 0x7ff6c00a3660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4100df0 tx=0x7ff6c8009260 comp rx=0 tx=0).stop 2024-05-08T22:36:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/2258724126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6d4101930 msgr2=0x7ff6d41a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6d4101930 0x7ff6d41a06c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc007b60 tx=0x7ff6cc008e90 comp rx=0 tx=0).stop 2024-05-08T22:36:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/2258724126 shutdown_connections 2024-05-08T22:36:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6d4101930 0x7ff6d41a06c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7ff6c00a11a0 0x7ff6c00a3660 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6d40ffda0 0x7ff6d4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 --2- 172.21.15.62:0/2258724126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d4100f90 0x7ff6d41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/2258724126 >> 172.21.15.62:0/2258724126 conn(0x7ff6d40fb330 msgr2=0x7ff6d40fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/2258724126 shutdown_connections 2024-05-08T22:36:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:31.736+0000 7ff6dc2f7700 1 -- 172.21.15.62:0/2258724126 wait complete. 2024-05-08T22:36:31.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:31 smithi066 ceph-mon[46830]: pgmap v264: 97 pgs: 97 active+clean; 662 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-08T22:36:31.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:31 smithi079 ceph-mon[46738]: pgmap v264: 97 pgs: 97 active+clean; 662 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-08T22:36:31.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:31 smithi062 ceph-mon[42571]: pgmap v264: 97 pgs: 97 active+clean; 662 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-08T22:36:33.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:33 smithi066 ceph-mon[46830]: pgmap v265: 97 pgs: 97 active+clean; 667 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-05-08T22:36:33.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:33 smithi079 ceph-mon[46738]: pgmap v265: 97 pgs: 97 active+clean; 667 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-05-08T22:36:33.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:33 smithi062 ceph-mon[42571]: pgmap v265: 97 pgs: 97 active+clean; 667 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-05-08T22:36:35.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:35 smithi066 ceph-mon[46830]: pgmap v266: 97 pgs: 97 active+clean; 667 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-05-08T22:36:35.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:35 smithi066 ceph-mon[46830]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-05-08T22:36:35.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:35 smithi079 ceph-mon[46738]: pgmap v266: 97 pgs: 97 active+clean; 667 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-05-08T22:36:35.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:35 smithi079 ceph-mon[46738]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-05-08T22:36:35.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:35 smithi062 ceph-mon[42571]: pgmap v266: 97 pgs: 97 active+clean; 667 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-05-08T22:36:35.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:35 smithi062 ceph-mon[42571]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-05-08T22:36:37.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:37 smithi066 ceph-mon[46830]: pgmap v267: 97 pgs: 97 active+clean; 667 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-05-08T22:36:37.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:37 smithi079 ceph-mon[46738]: pgmap v267: 97 pgs: 97 active+clean; 667 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-05-08T22:36:37.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:37 smithi062 ceph-mon[42571]: pgmap v267: 97 pgs: 97 active+clean; 667 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-05-08T22:36:39.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:39 smithi066 ceph-mon[46830]: pgmap v268: 97 pgs: 97 active+clean; 672 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 16 op/s 2024-05-08T22:36:39.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:39 smithi079 ceph-mon[46738]: pgmap v268: 97 pgs: 97 active+clean; 672 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 16 op/s 2024-05-08T22:36:39.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:39 smithi062 ceph-mon[42571]: pgmap v268: 97 pgs: 97 active+clean; 672 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 16 op/s 2024-05-08T22:36:40.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:40 smithi066 ceph-mon[46830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:36:40.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:40 smithi079 ceph-mon[46738]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:36:40.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:40 smithi062 ceph-mon[42571]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:36:41.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:41 smithi066 ceph-mon[46830]: pgmap v269: 97 pgs: 97 active+clean; 672 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-05-08T22:36:41.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:41 smithi079 ceph-mon[46738]: pgmap v269: 97 pgs: 97 active+clean; 672 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-05-08T22:36:41.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:41 smithi062 ceph-mon[42571]: pgmap v269: 97 pgs: 97 active+clean; 672 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-05-08T22:36:43.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:43 smithi066 ceph-mon[46830]: pgmap v270: 97 pgs: 97 active+clean; 680 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 16 op/s 2024-05-08T22:36:43.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:43 smithi079 ceph-mon[46738]: pgmap v270: 97 pgs: 97 active+clean; 680 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 16 op/s 2024-05-08T22:36:43.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:43 smithi062 ceph-mon[42571]: pgmap v270: 97 pgs: 97 active+clean; 680 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 16 op/s 2024-05-08T22:36:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.859+0000 7f56ae4ec700 1 -- 172.21.15.62:0/374154010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a80ffde0 msgr2=0x7f56a8100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.859+0000 7f56ae4ec700 1 --2- 172.21.15.62:0/374154010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a80ffde0 0x7f56a8100200 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f56980099c0 tx=0x7f5698009cd0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.859+0000 7f56ae4ec700 1 -- 172.21.15.62:0/374154010 shutdown_connections 2024-05-08T22:36:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.859+0000 7f56ae4ec700 1 --2- 172.21.15.62:0/374154010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a8101970 0x7f56a8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.859+0000 7f56ae4ec700 1 --2- 172.21.15.62:0/374154010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a80ffde0 0x7f56a8100200 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.859+0000 7f56ae4ec700 1 --2- 172.21.15.62:0/374154010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56a8100fd0 0x7f56a8101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.859+0000 7f56ae4ec700 1 -- 172.21.15.62:0/374154010 >> 172.21.15.62:0/374154010 conn(0x7f56a80fb370 msgr2=0x7f56a80fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.859+0000 7f56ae4ec700 1 -- 172.21.15.62:0/374154010 shutdown_connections 2024-05-08T22:36:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.859+0000 7f56ae4ec700 1 -- 172.21.15.62:0/374154010 wait complete. 2024-05-08T22:36:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56ae4ec700 1 Processor -- start 2024-05-08T22:36:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56ae4ec700 1 -- start start 2024-05-08T22:36:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56ae4ec700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a80ffde0 0x7f56a8196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56ae4ec700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56a8100fd0 0x7f56a8197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56ae4ec700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a8101970 0x7f56a819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56ae4ec700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a819e9e0 con 0x7f56a8101970 2024-05-08T22:36:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56ae4ec700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a819eb20 con 0x7f56a8100fd0 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56ae4ec700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a819ecb0 con 0x7f56a80ffde0 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56aca89700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a8101970 0x7f56a819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56a7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a80ffde0 0x7f56a8196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56aca89700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a8101970 0x7f56a819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52170/0 (socket says 172.21.15.62:52170) 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56aca89700 1 -- 172.21.15.62:0/2168524091 learned_addr learned my addr 172.21.15.62:0/2168524091 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.860+0000 7f56a77fe700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56a8100fd0 0x7f56a8197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.861+0000 7f56aca89700 1 -- 172.21.15.62:0/2168524091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a80ffde0 msgr2=0x7f56a8196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.861+0000 7f56aca89700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a80ffde0 0x7f56a8196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.861+0000 7f56aca89700 1 -- 172.21.15.62:0/2168524091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56a8100fd0 msgr2=0x7f56a8197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.861+0000 7f56aca89700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56a8100fd0 0x7f56a8197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.861+0000 7f56aca89700 1 -- 172.21.15.62:0/2168524091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56980095f0 con 0x7f56a8101970 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.861+0000 7f56aca89700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a8101970 0x7f56a819e410 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f56a0007ee0 tx=0x7f56a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.861+0000 7f56a57fa700 1 -- 172.21.15.62:0/2168524091 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0017070 con 0x7f56a8101970 2024-05-08T22:36:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.861+0000 7f56a57fa700 1 -- 172.21.15.62:0/2168524091 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56a000f040 con 0x7f56a8101970 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.861+0000 7f56ae4ec700 1 -- 172.21.15.62:0/2168524091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a819ef90 con 0x7f56a8101970 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.861+0000 7f56a57fa700 1 -- 172.21.15.62:0/2168524091 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0005580 con 0x7f56a8101970 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.861+0000 7f56ae4ec700 1 -- 172.21.15.62:0/2168524091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a819f410 con 0x7f56a8101970 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.863+0000 7f56a57fa700 1 -- 172.21.15.62:0/2168524091 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f56a000db20 con 0x7f56a8101970 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.864+0000 7f56a57fa700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f56900a1190 0x7f56900a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.864+0000 7f56a57fa700 1 -- 172.21.15.62:0/2168524091 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f56a00cbfb0 con 0x7f56a8101970 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.864+0000 7f56a7fff700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f56900a1190 0x7f56900a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56a7fff700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f56900a1190 0x7f56900a3650 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5698009f50 tx=0x7f5698005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56ba6a6b80 1 Processor -- start 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56ba6a6b80 1 -- start start 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed9753ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed9754cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed9755750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed96ab0f0 con 0x564ed97543e0 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed96a8ab0 con 0x564ed9753a80 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed96a43a0 con 0x564ed97552d0 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed9754cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed9754cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52174/0 (socket says 172.21.15.62:52174) 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56a77fe700 1 -- 172.21.15.62:0/3093941499 learned_addr learned my addr 172.21.15.62:0/3093941499 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56a7fff700 1 --2- 172.21.15.62:0/3093941499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed9753ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.865+0000 7f56aca89700 1 --2- 172.21.15.62:0/3093941499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed9755750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.866+0000 7f56a77fe700 1 -- 172.21.15.62:0/3093941499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 msgr2=0x564ed9755750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.866+0000 7f56a77fe700 1 --2- 172.21.15.62:0/3093941499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed9755750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.866+0000 7f56a77fe700 1 -- 172.21.15.62:0/3093941499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 msgr2=0x564ed9753ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.866+0000 7f56a77fe700 1 --2- 172.21.15.62:0/3093941499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed9753ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.866+0000 7f56a77fe700 1 -- 172.21.15.62:0/3093941499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ed9525350 con 0x564ed97543e0 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.866+0000 7f56a77fe700 1 --2- 172.21.15.62:0/3093941499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed9754cf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f569c00bd70 tx=0x7f569c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64297184a9134cc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.866+0000 7f568e7fc700 1 -- 172.21.15.62:0/3093941499 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c010e10 con 0x564ed97543e0 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.866+0000 7f568e7fc700 1 -- 172.21.15.62:0/3093941499 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f569c015070 con 0x564ed97543e0 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.866+0000 7f568e7fc700 1 -- 172.21.15.62:0/3093941499 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c011730 con 0x564ed97543e0 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.867+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3093941499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 msgr2=0x564ed9754cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.867+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3093941499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed9754cf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f569c00bd70 tx=0x7f569c0106b0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.867+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3093941499 shutdown_connections 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.867+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3093941499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed9755750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.867+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3093941499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed9753ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.867+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3093941499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed9754cf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.867+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3093941499 >> 172.21.15.62:0/3093941499 conn(0x564ed96b21b0 msgr2=0x564ed9750440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.867+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3093941499 shutdown_connections 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.867+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3093941499 wait complete. 2024-05-08T22:36:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.867+0000 7f56ba6a6b80 1 -- start start 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.867+0000 7f56ba6a6b80 1 Processor -- start 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed975d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed975ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed9764df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed9640500 con 0x564ed97543e0 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed96401a0 con 0x564ed9753a80 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed9527640 con 0x564ed97552d0 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56ba6a6b80 10 client.0 _ll_register_callbacks cb 0x564ed9649ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed975ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed975ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52178/0 (socket says 172.21.15.62:52178) 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56a77fe700 1 -- 172.21.15.62:0/3304534982 learned_addr learned my addr 172.21.15.62:0/3304534982 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56a7fff700 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed975d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56aca89700 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed9764df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56a77fe700 1 -- 172.21.15.62:0/3304534982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 msgr2=0x564ed9764df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56a77fe700 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed9764df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56a77fe700 1 -- 172.21.15.62:0/3304534982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 msgr2=0x564ed975d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56a77fe700 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed975d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56a77fe700 1 -- 172.21.15.62:0/3304534982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c00b9a0 con 0x564ed97543e0 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56aca89700 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed9764df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56a77fe700 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed975ddb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26157193fb9779 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.868+0000 7f56877fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56877fe700 1 -- 172.21.15.62:0/3304534982 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c01c890 con 0x564ed97543e0 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56877fe700 1 -- 172.21.15.62:0/3304534982 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f569c017070 con 0x564ed97543e0 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3304534982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ed95fd210 con 0x564ed97543e0 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56877fe700 1 -- 172.21.15.62:0/3304534982 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f569c004030 con 0x564ed97543e0 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56ba6a6b80 10 client.15399 fetch_fsmap learned FSMap version 29 2024-05-08T22:36:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3304534982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564ed94d1b00 con 0x564ed97543e0 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56877fe700 1 -- 172.21.15.62:0/3304534982 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ed95fd210 con 0x564ed97543e0 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56877fe700 1 -- 172.21.15.62:0/3304534982 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f569c003bc0 con 0x564ed97543e0 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56ba6a6b80 10 client.15399 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56ba6a6b80 10 client.15399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56ba6a6b80 20 client.15399 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x564ed97695c0 0x564ed976b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56ba6a6b80 4 client.15399 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3304534982 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ed97237b0 con 0x564ed97695c0 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.869+0000 7f56a7fff700 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x564ed97695c0 0x564ed976b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.870+0000 7f56a7fff700 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x564ed97695c0 0x564ed976b9e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90bdf48d8b0f796a server_cookie=17cd6cfd00275233 in_seq=0 out_seq=0 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.870+0000 7f56877fe700 10 client.15399 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.870+0000 7f56877fe700 1 -- 172.21.15.62:0/3304534982 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564ed97237b0 con 0x564ed97695c0 2024-05-08T22:36:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.870+0000 7f56877fe700 10 client.15399 handle_command_reply: tid=0 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 1 client.15399 shutdown 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3304534982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 msgr2=0x564ed975ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed975ddb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c010ec0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3304534982 shutdown_connections 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x564ed97695c0 0x564ed976b9e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed9764df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed975d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56877fe700 0 client.15399 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3304534982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97543e0 0x564ed975ddb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3304534982 >> 172.21.15.62:0/3304534982 conn(0x564ed96b21b0 msgr2=0x564ed974e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3304534982 shutdown_connections 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3304534982 wait complete. 2024-05-08T22:36:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.871+0000 7f56ba6a6b80 20 client.15399 trim_cache size 0 max 16384 2024-05-08T22:36:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.890+0000 7f56ba6a6b80 1 Processor -- start 2024-05-08T22:36:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.890+0000 7f56ba6a6b80 1 -- start start 2024-05-08T22:36:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.890+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed9753a80 0x564ed9753e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.890+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed97543e0 0x564ed9764db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.890+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed97654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.890+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed9770c30 con 0x564ed9753a80 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.890+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed97547f0 con 0x564ed97543e0 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.890+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed975e4f0 con 0x564ed97552d0 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f56a7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed9753a80 0x564ed9753e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f56a7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed9753a80 0x564ed9753e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52184/0 (socket says 172.21.15.62:52184) 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f56a7fff700 1 -- 172.21.15.62:0/3627493247 learned_addr learned my addr 172.21.15.62:0/3627493247 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f56a77fe700 1 --2- 172.21.15.62:0/3627493247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed97543e0 0x564ed9764db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f56aca89700 1 --2- 172.21.15.62:0/3627493247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed97654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f56a7fff700 1 -- 172.21.15.62:0/3627493247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 msgr2=0x564ed97654a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f56a7fff700 1 --2- 172.21.15.62:0/3627493247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed97654a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f56a7fff700 1 -- 172.21.15.62:0/3627493247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed97543e0 msgr2=0x564ed9764db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f56a7fff700 1 --2- 172.21.15.62:0/3627493247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed97543e0 0x564ed9764db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f56a7fff700 1 -- 172.21.15.62:0/3627493247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ed966a500 con 0x564ed9753a80 2024-05-08T22:36:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f56a7fff700 1 --2- 172.21.15.62:0/3627493247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed9753a80 0x564ed9753e60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f56980099c0 tx=0x7f5698023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc1d90eac1e8d3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f568effd700 1 -- 172.21.15.62:0/3627493247 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698036160 con 0x564ed9753a80 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f568effd700 1 -- 172.21.15.62:0/3627493247 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f569801e040 con 0x564ed9753a80 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.891+0000 7f568effd700 1 -- 172.21.15.62:0/3627493247 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698003e70 con 0x564ed9753a80 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.892+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3627493247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed9753a80 msgr2=0x564ed9753e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.892+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3627493247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed9753a80 0x564ed9753e60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f56980099c0 tx=0x7f5698023000 comp rx=0 tx=0).stop 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.892+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3627493247 shutdown_connections 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.892+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3627493247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97552d0 0x564ed97654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.892+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3627493247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed9753a80 0x564ed9753e60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.892+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/3627493247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed97543e0 0x564ed9764db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.892+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3627493247 >> 172.21.15.62:0/3627493247 conn(0x564ed96b21b0 msgr2=0x564ed975d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.892+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3627493247 shutdown_connections 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.892+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/3627493247 wait complete. 2024-05-08T22:36:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.892+0000 7f56ba6a6b80 1 -- start start 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.892+0000 7f56ba6a6b80 1 Processor -- start 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed97622d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97543e0 0x564ed9762810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56ba6a6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97552d0 0x564ed97636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed9763c20 con 0x564ed97552d0 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed9763d90 con 0x564ed9753a80 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56ba6a6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed9763f50 con 0x564ed97543e0 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56ba6a6b80 10 client.0 _ll_register_callbacks cb 0x564ed9649ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56aca89700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97552d0 0x564ed97636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56aca89700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97552d0 0x564ed97636e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52196/0 (socket says 172.21.15.62:52196) 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56aca89700 1 -- 172.21.15.62:0/2894987724 learned_addr learned my addr 172.21.15.62:0/2894987724 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56a7fff700 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed97622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56aca89700 1 -- 172.21.15.62:0/2894987724 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97543e0 msgr2=0x564ed9762810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56aca89700 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97543e0 0x564ed9762810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56aca89700 1 -- 172.21.15.62:0/2894987724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 msgr2=0x564ed97622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56aca89700 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed97622d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56aca89700 1 -- 172.21.15.62:0/2894987724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a8108b70 con 0x564ed97552d0 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56aca89700 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97552d0 0x564ed97636e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f56a0095910 tx=0x7f56a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818e61f84f4c8a1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56877fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56877fe700 1 -- 172.21.15.62:0/2894987724 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a009d240 con 0x564ed97552d0 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.893+0000 7f56877fe700 1 -- 172.21.15.62:0/2894987724 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f56a00a1890 con 0x564ed97552d0 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/2894987724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f569c015070 con 0x564ed97552d0 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56877fe700 1 -- 172.21.15.62:0/2894987724 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56a00977b0 con 0x564ed97552d0 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56ba6a6b80 10 client.15405 fetch_fsmap learned FSMap version 29 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/2894987724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564ed9529c70 con 0x564ed97552d0 2024-05-08T22:36:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56877fe700 1 -- 172.21.15.62:0/2894987724 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f569c015070 con 0x564ed97552d0 2024-05-08T22:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56877fe700 1 -- 172.21.15.62:0/2894987724 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f56a0098900 con 0x564ed97552d0 2024-05-08T22:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56ba6a6b80 10 client.15405 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56ba6a6b80 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56ba6a6b80 20 client.15405 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x564ed97c0150 0x564ed97c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56ba6a6b80 4 client.15405 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/2894987724 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564ed969ce60 con 0x564ed97c0150 2024-05-08T22:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.894+0000 7f56a7fff700 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x564ed97c0150 0x564ed97c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.895+0000 7f56877fe700 10 client.15405 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.895+0000 7f56a7fff700 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x564ed97c0150 0x564ed97c2570 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff933f4d39f8cd32 server_cookie=fd8e6e6e2db24daa in_seq=0 out_seq=0 2024-05-08T22:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56877fe700 1 -- 172.21.15.62:0/2894987724 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564ed969ce60 con 0x564ed97c0150 2024-05-08T22:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56877fe700 10 client.15405 handle_command_reply: tid=0 2024-05-08T22:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56ba6a6b80 1 client.15405 shutdown 2024-05-08T22:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/2894987724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97552d0 msgr2=0x564ed97636e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97552d0 0x564ed97636e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f56a0095910 tx=0x7f56a000b040 comp rx=0 tx=0).stop 2024-05-08T22:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/2894987724 shutdown_connections 2024-05-08T22:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x564ed97c0150 0x564ed97c2570 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564ed97552d0 0x564ed97636e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ed9753a80 0x564ed97622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56877fe700 0 client.15405 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56ba6a6b80 1 --2- 172.21.15.62:0/2894987724 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564ed97543e0 0x564ed9762810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/2894987724 >> 172.21.15.62:0/2894987724 conn(0x564ed96b21b0 msgr2=0x564ed975d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.902+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/2894987724 shutdown_connections 2024-05-08T22:36:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ba6a6b80 1 -- 172.21.15.62:0/2894987724 wait complete. 2024-05-08T22:36:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ba6a6b80 20 client.15405 trim_cache size 0 max 16384 2024-05-08T22:36:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-05-08T22:36:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-05-08T22:36:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-05-08T22:36:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-05-08T22:36:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 -- 172.21.15.62:0/2168524091 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f56900a1190 msgr2=0x7f56900a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f56900a1190 0x7f56900a3650 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5698009f50 tx=0x7f5698005ea0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 -- 172.21.15.62:0/2168524091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a8101970 msgr2=0x7f56a819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a8101970 0x7f56a819e410 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f56a0007ee0 tx=0x7f56a000d3b0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 -- 172.21.15.62:0/2168524091 shutdown_connections 2024-05-08T22:36:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a8101970 0x7f56a819e410 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a80ffde0 0x7f56a8196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f56900a1190 0x7f56900a3650 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 --2- 172.21.15.62:0/2168524091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56a8100fd0 0x7f56a8197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 -- 172.21.15.62:0/2168524091 >> 172.21.15.62:0/2168524091 conn(0x7f56a80fb370 msgr2=0x7f56a80fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 -- 172.21.15.62:0/2168524091 shutdown_connections 2024-05-08T22:36:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:44.903+0000 7f56ae4ec700 1 -- 172.21.15.62:0/2168524091 wait complete. 2024-05-08T22:36:45.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:45 smithi066 ceph-mon[46830]: pgmap v271: 97 pgs: 97 active+clean; 680 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-05-08T22:36:45.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:45 smithi079 ceph-mon[46738]: pgmap v271: 97 pgs: 97 active+clean; 680 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-05-08T22:36:45.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:45 smithi062 ceph-mon[42571]: pgmap v271: 97 pgs: 97 active+clean; 680 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-05-08T22:36:47.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:47 smithi066 ceph-mon[46830]: pgmap v272: 97 pgs: 97 active+clean; 680 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-08T22:36:47.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:47 smithi079 ceph-mon[46738]: pgmap v272: 97 pgs: 97 active+clean; 680 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-08T22:36:47.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:47 smithi062 ceph-mon[42571]: pgmap v272: 97 pgs: 97 active+clean; 680 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-08T22:36:49.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:49 smithi066 ceph-mon[46830]: pgmap v273: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-05-08T22:36:49.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:49 smithi079 ceph-mon[46738]: pgmap v273: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-05-08T22:36:49.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:49 smithi062 ceph-mon[42571]: pgmap v273: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-05-08T22:36:51.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:51 smithi066 ceph-mon[46830]: pgmap v274: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-08T22:36:51.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:51 smithi079 ceph-mon[46738]: pgmap v274: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-08T22:36:51.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:51 smithi062 ceph-mon[42571]: pgmap v274: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-08T22:36:53.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:53 smithi066 ceph-mon[46830]: pgmap v275: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-05-08T22:36:53.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:53 smithi079 ceph-mon[46738]: pgmap v275: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-05-08T22:36:53.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:53 smithi062 ceph-mon[42571]: pgmap v275: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-05-08T22:36:55.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:55 smithi066 ceph-mon[46830]: pgmap v276: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-08T22:36:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:55 smithi079 ceph-mon[46738]: pgmap v276: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-08T22:36:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:55 smithi062 ceph-mon[42571]: pgmap v276: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-08T22:36:57.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:57 smithi066 ceph-mon[46830]: pgmap v277: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-08T22:36:57.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:57 smithi079 ceph-mon[46738]: pgmap v277: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-08T22:36:57.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:57 smithi062 ceph-mon[42571]: pgmap v277: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-08T22:36:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.028+0000 7f60b467b700 1 -- 172.21.15.62:0/3246586390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ac102040 msgr2=0x7f60ac1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.028+0000 7f60b467b700 1 --2- 172.21.15.62:0/3246586390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ac102040 0x7f60ac1024c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f609c009990 tx=0x7f609c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.028+0000 7f60b467b700 1 -- 172.21.15.62:0/3246586390 shutdown_connections 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.028+0000 7f60b467b700 1 --2- 172.21.15.62:0/3246586390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60ac102a00 0x7f60ac109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.028+0000 7f60b467b700 1 --2- 172.21.15.62:0/3246586390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60ac100e60 0x7f60ac101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.028+0000 7f60b467b700 1 --2- 172.21.15.62:0/3246586390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ac102040 0x7f60ac1024c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.028+0000 7f60b467b700 1 -- 172.21.15.62:0/3246586390 >> 172.21.15.62:0/3246586390 conn(0x7f60ac0fc460 msgr2=0x7f60ac0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.028+0000 7f60b467b700 1 -- 172.21.15.62:0/3246586390 shutdown_connections 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.028+0000 7f60b467b700 1 -- 172.21.15.62:0/3246586390 wait complete. 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.028+0000 7f60b467b700 1 Processor -- start 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b467b700 1 -- start start 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b467b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60ac100e60 0x7f60ac074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b467b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60ac102040 0x7f60ac073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b467b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ac102a00 0x7f60ac0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b467b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac073fd0 con 0x7f60ac102a00 2024-05-08T22:36:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b467b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac074140 con 0x7f60ac102040 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b467b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac1a0eb0 con 0x7f60ac100e60 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b2c18700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ac102a00 0x7f60ac0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b2c18700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ac102a00 0x7f60ac0738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52776/0 (socket says 172.21.15.62:52776) 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b2c18700 1 -- 172.21.15.62:0/344187186 learned_addr learned my addr 172.21.15.62:0/344187186 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b1c16700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60ac102040 0x7f60ac073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b2417700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60ac100e60 0x7f60ac074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b2c18700 1 -- 172.21.15.62:0/344187186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60ac100e60 msgr2=0x7f60ac074cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b2c18700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60ac100e60 0x7f60ac074cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b2c18700 1 -- 172.21.15.62:0/344187186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60ac102040 msgr2=0x7f60ac073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b2c18700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60ac102040 0x7f60ac073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b2c18700 1 -- 172.21.15.62:0/344187186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c0095f0 con 0x7f60ac102a00 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.029+0000 7f60b1c16700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60ac102040 0x7f60ac073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.030+0000 7f60b2c18700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ac102a00 0x7f60ac0738d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f60a4007ee0 tx=0x7f60a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.030+0000 7f60a37fe700 1 -- 172.21.15.62:0/344187186 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a400dd20 con 0x7f60ac102a00 2024-05-08T22:36:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.030+0000 7f60a37fe700 1 -- 172.21.15.62:0/344187186 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a400f040 con 0x7f60ac102a00 2024-05-08T22:36:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.030+0000 7f60b467b700 1 -- 172.21.15.62:0/344187186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac1a1110 con 0x7f60ac102a00 2024-05-08T22:36:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.030+0000 7f60a37fe700 1 -- 172.21.15.62:0/344187186 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a40048a0 con 0x7f60ac102a00 2024-05-08T22:36:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.030+0000 7f60b467b700 1 -- 172.21.15.62:0/344187186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac1a15d0 con 0x7f60ac102a00 2024-05-08T22:36:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.032+0000 7f60a37fe700 1 -- 172.21.15.62:0/344187186 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f60a4003a00 con 0x7f60ac102a00 2024-05-08T22:36:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.032+0000 7f60a37fe700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60980a1110 0x7f60980a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.033+0000 7f60a37fe700 1 -- 172.21.15.62:0/344187186 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f60a40cc120 con 0x7f60ac102a00 2024-05-08T22:36:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.033+0000 7f60b2417700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60980a1110 0x7f60980a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.033+0000 7f60b2417700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60980a1110 0x7f60980a35d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f60ac101bb0 tx=0x7f60a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.033+0000 7f60c0835b80 1 Processor -- start 2024-05-08T22:36:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60c0835b80 1 -- start start 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dedeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41dedfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41dee0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60c0835b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41de360f0 con 0x55e41dedea80 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60c0835b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41de33ab0 con 0x55e41dee02d0 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60c0835b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41de2f3a0 con 0x55e41dedf3e0 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b2417700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dedeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b2417700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dedeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52778/0 (socket says 172.21.15.62:52778) 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b2417700 1 -- 172.21.15.62:0/2265905039 learned_addr learned my addr 172.21.15.62:0/2265905039 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b1c16700 1 --2- 172.21.15.62:0/2265905039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41dedfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b2c18700 1 --2- 172.21.15.62:0/2265905039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41dee0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b2417700 1 -- 172.21.15.62:0/2265905039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 msgr2=0x55e41dedfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b2417700 1 --2- 172.21.15.62:0/2265905039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41dedfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b2417700 1 -- 172.21.15.62:0/2265905039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 msgr2=0x55e41dee0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b2417700 1 --2- 172.21.15.62:0/2265905039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41dee0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b2417700 1 -- 172.21.15.62:0/2265905039 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e41dcb0350 con 0x55e41dedea80 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b2c18700 1 --2- 172.21.15.62:0/2265905039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41dee0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.034+0000 7f60b2417700 1 --2- 172.21.15.62:0/2265905039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dedeea0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f60a8011d50 tx=0x7f60a8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc8fd8d89d9574e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.035+0000 7f60a0ff9700 1 -- 172.21.15.62:0/2265905039 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a800ed20 con 0x55e41dedea80 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.035+0000 7f60a0ff9700 1 -- 172.21.15.62:0/2265905039 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a8021070 con 0x55e41dedea80 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.035+0000 7f60a0ff9700 1 -- 172.21.15.62:0/2265905039 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a801d670 con 0x55e41dedea80 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.035+0000 7f60c0835b80 1 -- 172.21.15.62:0/2265905039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 msgr2=0x55e41dedeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.035+0000 7f60c0835b80 1 --2- 172.21.15.62:0/2265905039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dedeea0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f60a8011d50 tx=0x7f60a8014000 comp rx=0 tx=0).stop 2024-05-08T22:36:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.035+0000 7f60c0835b80 1 -- 172.21.15.62:0/2265905039 shutdown_connections 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.035+0000 7f60c0835b80 1 --2- 172.21.15.62:0/2265905039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41dee0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.035+0000 7f60c0835b80 1 --2- 172.21.15.62:0/2265905039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dedeea0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.035+0000 7f60c0835b80 1 --2- 172.21.15.62:0/2265905039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41dedfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.035+0000 7f60c0835b80 1 -- 172.21.15.62:0/2265905039 >> 172.21.15.62:0/2265905039 conn(0x55e41de3d1b0 msgr2=0x55e41dedb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60c0835b80 1 -- 172.21.15.62:0/2265905039 shutdown_connections 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60c0835b80 1 -- 172.21.15.62:0/2265905039 wait complete. 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60c0835b80 1 -- start start 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60c0835b80 1 Processor -- start 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dee8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41dee8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41deefdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60c0835b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41ddcb500 con 0x55e41dedea80 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60c0835b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41ddcb1a0 con 0x55e41dee02d0 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60c0835b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41dcb2640 con 0x55e41dedf3e0 2024-05-08T22:36:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60c0835b80 10 client.0 _ll_register_callbacks cb 0x55e41ddd4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60b2417700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dee8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60b2417700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dee8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52782/0 (socket says 172.21.15.62:52782) 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.036+0000 7f60b2417700 1 -- 172.21.15.62:0/3796122051 learned_addr learned my addr 172.21.15.62:0/3796122051 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60b2c18700 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41deefdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60b1c16700 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41dee8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60b2417700 1 -- 172.21.15.62:0/3796122051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 msgr2=0x55e41dee8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60b2417700 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41dee8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60b2417700 1 -- 172.21.15.62:0/3796122051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 msgr2=0x55e41deefdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60b2417700 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41deefdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60b2417700 1 -- 172.21.15.62:0/3796122051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a8017070 con 0x55e41dedea80 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60b2c18700 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41deefdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60b2417700 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dee8830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f60a801cc90 tx=0x7f60a800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e1b8a880c22b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f6095ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f6095ffb700 1 -- 172.21.15.62:0/3796122051 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a801c410 con 0x55e41dedea80 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f6095ffb700 1 -- 172.21.15.62:0/3796122051 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f60a801c570 con 0x55e41dedea80 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60c0835b80 1 -- 172.21.15.62:0/3796122051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e41dd88210 con 0x55e41dedea80 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f6095ffb700 1 -- 172.21.15.62:0/3796122051 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60a801c730 con 0x55e41dedea80 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60c0835b80 10 client.15417 fetch_fsmap learned FSMap version 29 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.037+0000 7f60c0835b80 1 -- 172.21.15.62:0/3796122051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e41dc5cb00 con 0x55e41dedea80 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.038+0000 7f6095ffb700 1 -- 172.21.15.62:0/3796122051 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a803d020 con 0x55e41dedea80 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.038+0000 7f6095ffb700 1 -- 172.21.15.62:0/3796122051 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60a8004a30 con 0x55e41dedea80 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.038+0000 7f60c0835b80 10 client.15417 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.038+0000 7f60c0835b80 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.038+0000 7f60c0835b80 20 client.15417 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.038+0000 7f60c0835b80 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55e41def4630 0x55e41def6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.038+0000 7f60c0835b80 4 client.15417 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.038+0000 7f60c0835b80 1 -- 172.21.15.62:0/3796122051 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e41deae7b0 con 0x55e41def4630 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.038+0000 7f60b1c16700 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55e41def4630 0x55e41def6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f6095ffb700 10 client.15417 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60b1c16700 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55e41def4630 0x55e41def6a50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbcb82c56cc2b201 server_cookie=d4743b2ce3e48c6c in_seq=0 out_seq=0 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f6095ffb700 1 -- 172.21.15.62:0/3796122051 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e41deae7b0 con 0x55e41def4630 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f6095ffb700 10 client.15417 handle_command_reply: tid=0 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60c0835b80 1 client.15417 shutdown 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60c0835b80 1 -- 172.21.15.62:0/3796122051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 msgr2=0x55e41dee8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60c0835b80 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dee8830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f60a801cc90 tx=0x7f60a800eb20 comp rx=0 tx=0).stop 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60c0835b80 1 -- 172.21.15.62:0/3796122051 shutdown_connections 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60c0835b80 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55e41def4630 0x55e41def6a50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60c0835b80 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41deefdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60c0835b80 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedea80 0x55e41dee8830 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60c0835b80 1 --2- 172.21.15.62:0/3796122051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41dee8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f6095ffb700 0 client.15417 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60c0835b80 1 -- 172.21.15.62:0/3796122051 >> 172.21.15.62:0/3796122051 conn(0x55e41de3d1b0 msgr2=0x55e41ded9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60c0835b80 1 -- 172.21.15.62:0/3796122051 shutdown_connections 2024-05-08T22:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.039+0000 7f60c0835b80 1 -- 172.21.15.62:0/3796122051 wait complete. 2024-05-08T22:36:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.040+0000 7f60c0835b80 20 client.15417 trim_cache size 0 max 16384 2024-05-08T22:36:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60c0835b80 1 Processor -- start 2024-05-08T22:36:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60c0835b80 1 -- start start 2024-05-08T22:36:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedea80 0x55e41def6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedf3e0 0x55e41dedee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41def0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60c0835b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41deda930 con 0x55e41dedf3e0 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60c0835b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41defbc20 con 0x55e41dee02d0 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60c0835b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41dee3d50 con 0x55e41dedea80 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60b1c16700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedf3e0 0x55e41dedee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60b1c16700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedf3e0 0x55e41dedee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52784/0 (socket says 172.21.15.62:52784) 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60b1c16700 1 -- 172.21.15.62:0/222147579 learned_addr learned my addr 172.21.15.62:0/222147579 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60b2c18700 1 --2- 172.21.15.62:0/222147579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41def0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60b1c16700 1 -- 172.21.15.62:0/222147579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedea80 msgr2=0x55e41def6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60b1c16700 1 --2- 172.21.15.62:0/222147579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedea80 0x55e41def6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60b1c16700 1 -- 172.21.15.62:0/222147579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 msgr2=0x55e41def0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60b1c16700 1 --2- 172.21.15.62:0/222147579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41def0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60b1c16700 1 -- 172.21.15.62:0/222147579 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e41ddf5500 con 0x55e41dedf3e0 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.055+0000 7f60b1c16700 1 --2- 172.21.15.62:0/222147579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedf3e0 0x55e41dedee60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f609c00b2e0 tx=0x7f609c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40430d1ce318a840 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60a17fa700 1 -- 172.21.15.62:0/222147579 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c01d070 con 0x55e41dedf3e0 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60a17fa700 1 -- 172.21.15.62:0/222147579 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f609c01e040 con 0x55e41dedf3e0 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60a17fa700 1 -- 172.21.15.62:0/222147579 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c00fa50 con 0x55e41dedf3e0 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60c0835b80 1 -- 172.21.15.62:0/222147579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedf3e0 msgr2=0x55e41dedee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60c0835b80 1 --2- 172.21.15.62:0/222147579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedf3e0 0x55e41dedee60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f609c00b2e0 tx=0x7f609c009310 comp rx=0 tx=0).stop 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60c0835b80 1 -- 172.21.15.62:0/222147579 shutdown_connections 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60c0835b80 1 --2- 172.21.15.62:0/222147579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dee02d0 0x55e41def0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60c0835b80 1 --2- 172.21.15.62:0/222147579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedea80 0x55e41def6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60c0835b80 1 --2- 172.21.15.62:0/222147579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dedf3e0 0x55e41dedee60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60c0835b80 1 -- 172.21.15.62:0/222147579 >> 172.21.15.62:0/222147579 conn(0x55e41de3d1b0 msgr2=0x55e41dee67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60c0835b80 1 -- 172.21.15.62:0/222147579 shutdown_connections 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.056+0000 7f60c0835b80 1 -- 172.21.15.62:0/222147579 wait complete. 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60c0835b80 1 -- start start 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60c0835b80 1 Processor -- start 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dedea80 0x55e41deed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41deed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60c0835b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dee02d0 0x55e41deee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60c0835b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41deeec30 con 0x55e41dee02d0 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60c0835b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41deeeda0 con 0x55e41dedea80 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60c0835b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41deeef60 con 0x55e41dedf3e0 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60c0835b80 10 client.0 _ll_register_callbacks cb 0x55e41ddd4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60b2c18700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dee02d0 0x55e41deee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60b2c18700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dee02d0 0x55e41deee6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52788/0 (socket says 172.21.15.62:52788) 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60b2c18700 1 -- 172.21.15.62:0/439576377 learned_addr learned my addr 172.21.15.62:0/439576377 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:36:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60b2417700 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dedea80 0x55e41deed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60b2c18700 1 -- 172.21.15.62:0/439576377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 msgr2=0x55e41deed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60b1c16700 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41deed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60b2c18700 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41deed730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60b2c18700 1 -- 172.21.15.62:0/439576377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dedea80 msgr2=0x55e41deed1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60b2c18700 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dedea80 0x55e41deed1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.057+0000 7f60b2c18700 1 -- 172.21.15.62:0/439576377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac109c50 con 0x55e41dee02d0 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f60b2c18700 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dee02d0 0x55e41deee6f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f60a40c9c30 tx=0x7f60a4099190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ae02c55bba5107 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f6095ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f6095ffb700 1 -- 172.21.15.62:0/439576377 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a40b3030 con 0x55e41dee02d0 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f6095ffb700 1 -- 172.21.15.62:0/439576377 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f60a40b4070 con 0x55e41dee02d0 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f60c0835b80 1 -- 172.21.15.62:0/439576377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60a8021070 con 0x55e41dee02d0 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f6095ffb700 1 -- 172.21.15.62:0/439576377 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60a409d3b0 con 0x55e41dee02d0 2024-05-08T22:36:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f60c0835b80 10 client.15423 fetch_fsmap learned FSMap version 29 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f60c0835b80 1 -- 172.21.15.62:0/439576377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e41dcb4c70 con 0x55e41dee02d0 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f6095ffb700 1 -- 172.21.15.62:0/439576377 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a8021070 con 0x55e41dee02d0 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f6095ffb700 1 -- 172.21.15.62:0/439576377 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60a4099870 con 0x55e41dee02d0 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f60c0835b80 10 client.15423 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f60c0835b80 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f60c0835b80 20 client.15423 populate_metadata read hostname 'smithi062' 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f60c0835b80 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55e41df4b1d0 0x55e41df4d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f60c0835b80 4 client.15423 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.058+0000 7f60c0835b80 1 -- 172.21.15.62:0/439576377 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e41de27e60 con 0x55e41df4b1d0 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.059+0000 7f60b2417700 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55e41df4b1d0 0x55e41df4d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.059+0000 7f6095ffb700 10 client.15423 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.059+0000 7f60b2417700 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55e41df4b1d0 0x55e41df4d5f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b212bcd9113020 server_cookie=5f91ced6bcff37c7 in_seq=0 out_seq=0 2024-05-08T22:36:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f6095ffb700 1 -- 172.21.15.62:0/439576377 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e41de27e60 con 0x55e41df4b1d0 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f6095ffb700 10 client.15423 handle_command_reply: tid=0 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 1 client.15423 shutdown 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 1 -- 172.21.15.62:0/439576377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dee02d0 msgr2=0x55e41deee6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dee02d0 0x55e41deee6f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f60a40c9c30 tx=0x7f60a4099190 comp rx=0 tx=0).stop 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 1 -- 172.21.15.62:0/439576377 shutdown_connections 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55e41df4b1d0 0x55e41df4d5f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e41dee02d0 0x55e41deee6f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e41dedea80 0x55e41deed1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 1 --2- 172.21.15.62:0/439576377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e41dedf3e0 0x55e41deed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f6095ffb700 0 client.15423 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 1 -- 172.21.15.62:0/439576377 >> 172.21.15.62:0/439576377 conn(0x55e41de3d1b0 msgr2=0x55e41dee6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 1 -- 172.21.15.62:0/439576377 shutdown_connections 2024-05-08T22:36:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 1 -- 172.21.15.62:0/439576377 wait complete. 2024-05-08T22:36:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.063+0000 7f60c0835b80 20 client.15423 trim_cache size 0 max 16384 2024-05-08T22:36:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-05-08T22:36:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-05-08T22:36:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-05-08T22:36:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-05-08T22:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 -- 172.21.15.62:0/344187186 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60980a1110 msgr2=0x7f60980a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60980a1110 0x7f60980a35d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f60ac101bb0 tx=0x7f60a8009260 comp rx=0 tx=0).stop 2024-05-08T22:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 -- 172.21.15.62:0/344187186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ac102a00 msgr2=0x7f60ac0738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ac102a00 0x7f60ac0738d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f60a4007ee0 tx=0x7f60a400d3b0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 -- 172.21.15.62:0/344187186 shutdown_connections 2024-05-08T22:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ac102a00 0x7f60ac0738d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60ac100e60 0x7f60ac074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f60980a1110 0x7f60980a35d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 --2- 172.21.15.62:0/344187186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60ac102040 0x7f60ac073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 -- 172.21.15.62:0/344187186 >> 172.21.15.62:0/344187186 conn(0x7f60ac0fc460 msgr2=0x7f60ac1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 -- 172.21.15.62:0/344187186 shutdown_connections 2024-05-08T22:36:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:36:58.064+0000 7f60b467b700 1 -- 172.21.15.62:0/344187186 wait complete. 2024-05-08T22:36:59.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:36:59 smithi066 ceph-mon[46830]: pgmap v278: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-08T22:36:59.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:36:59 smithi079 ceph-mon[46738]: pgmap v278: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-08T22:36:59.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:36:59 smithi062 ceph-mon[42571]: pgmap v278: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-08T22:37:01.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:01 smithi066 ceph-mon[46830]: pgmap v279: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-08T22:37:01.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:01 smithi079 ceph-mon[46738]: pgmap v279: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-08T22:37:01.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:01 smithi062 ceph-mon[42571]: pgmap v279: 97 pgs: 97 active+clean; 685 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-08T22:37:03.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:03 smithi066 ceph-mon[46830]: pgmap v280: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:03.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:03 smithi079 ceph-mon[46738]: pgmap v280: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:03.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:03 smithi062 ceph-mon[42571]: pgmap v280: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:05.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:05 smithi066 ceph-mon[46830]: pgmap v281: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:05.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:05 smithi079 ceph-mon[46738]: pgmap v281: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:05.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:05 smithi062 ceph-mon[42571]: pgmap v281: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:07.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:07 smithi066 ceph-mon[46830]: pgmap v282: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:07.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:07 smithi079 ceph-mon[46738]: pgmap v282: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:07.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:07 smithi062 ceph-mon[42571]: pgmap v282: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:09.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:09 smithi066 ceph-mon[46830]: pgmap v283: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:09.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:09 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:37:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:09 smithi079 ceph-mon[46738]: pgmap v283: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:09 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:37:09.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:09 smithi062 ceph-mon[42571]: pgmap v283: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:09.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:09 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.183+0000 7f657fec9700 1 -- 172.21.15.62:0/328449256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65780ff520 msgr2=0x7f65780ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.183+0000 7f657fec9700 1 --2- 172.21.15.62:0/328449256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65780ff520 0x7f65780ff940 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f65740099c0 tx=0x7f6574009cd0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.183+0000 7f657fec9700 1 -- 172.21.15.62:0/328449256 shutdown_connections 2024-05-08T22:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.183+0000 7f657fec9700 1 --2- 172.21.15.62:0/328449256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65781010b0 0x7f6578107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.183+0000 7f657fec9700 1 --2- 172.21.15.62:0/328449256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65780ff520 0x7f65780ff940 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.183+0000 7f657fec9700 1 --2- 172.21.15.62:0/328449256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6578100710 0x7f6578100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.183+0000 7f657fec9700 1 -- 172.21.15.62:0/328449256 >> 172.21.15.62:0/328449256 conn(0x7f65780faab0 msgr2=0x7f65780fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.183+0000 7f657fec9700 1 -- 172.21.15.62:0/328449256 shutdown_connections 2024-05-08T22:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.183+0000 7f657fec9700 1 -- 172.21.15.62:0/328449256 wait complete. 2024-05-08T22:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657fec9700 1 Processor -- start 2024-05-08T22:37:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657fec9700 1 -- start start 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657fec9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65780ff520 0x7f6578199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657fec9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6578100710 0x7f65781995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657fec9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65781010b0 0x7f65781a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657fec9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65781a0bb0 con 0x7f65781010b0 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657fec9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65781a0cf0 con 0x7f6578100710 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657fec9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65781a0e80 con 0x7f65780ff520 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657e466700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65781010b0 0x7f65781a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657dc65700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65780ff520 0x7f6578199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657dc65700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65780ff520 0x7f6578199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.62:54280/0 (socket says 172.21.15.62:54280) 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657d464700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6578100710 0x7f65781995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657dc65700 1 -- 172.21.15.62:0/2049337662 learned_addr learned my addr 172.21.15.62:0/2049337662 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657dc65700 1 -- 172.21.15.62:0/2049337662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6578100710 msgr2=0x7f65781995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657dc65700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6578100710 0x7f65781995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657dc65700 1 -- 172.21.15.62:0/2049337662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65781010b0 msgr2=0x7f65781a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657dc65700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65781010b0 0x7f65781a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.184+0000 7f657dc65700 1 -- 172.21.15.62:0/2049337662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6570009c40 con 0x7f65780ff520 2024-05-08T22:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.185+0000 7f657dc65700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65780ff520 0x7f6578199080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6574003730 tx=0x7f657400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.185+0000 7f656effd700 1 -- 172.21.15.62:0/2049337662 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657401d070 con 0x7f65780ff520 2024-05-08T22:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.185+0000 7f657fec9700 1 -- 172.21.15.62:0/2049337662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65740095f0 con 0x7f65780ff520 2024-05-08T22:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.185+0000 7f656effd700 1 -- 172.21.15.62:0/2049337662 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6574005050 con 0x7f65780ff520 2024-05-08T22:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.185+0000 7f656effd700 1 -- 172.21.15.62:0/2049337662 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574004a30 con 0x7f65780ff520 2024-05-08T22:37:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.185+0000 7f657fec9700 1 -- 172.21.15.62:0/2049337662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65781a13f0 con 0x7f65780ff520 2024-05-08T22:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.187+0000 7f656effd700 1 -- 172.21.15.62:0/2049337662 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f65740051c0 con 0x7f65780ff520 2024-05-08T22:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.188+0000 7f656effd700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65640a1110 0x7f65640a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.188+0000 7f657d464700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65640a1110 0x7f65640a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.188+0000 7f656effd700 1 -- 172.21.15.62:0/2049337662 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f65740cd5f0 con 0x7f65780ff520 2024-05-08T22:37:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.189+0000 7f657d464700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65640a1110 0x7f65640a35d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f656800acd0 tx=0x7f6568010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.189+0000 7f658c083b80 1 Processor -- start 2024-05-08T22:37:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.189+0000 7f658c083b80 1 -- start start 2024-05-08T22:37:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.189+0000 7f658c083b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 0x559100fb8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.189+0000 7f658c083b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb93e0 0x559100fb9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.189+0000 7f658c083b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 0x559100fba750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.189+0000 7f658c083b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100f100f0 con 0x559100fba2d0 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f658c083b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100f0dab0 con 0x559100fb8a80 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f658c083b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100f093a0 con 0x559100fb93e0 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f657dc65700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 0x559100fb8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f657dc65700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 0x559100fb8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.62:45938/0 (socket says 172.21.15.62:45938) 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f657e466700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 0x559100fba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f657dc65700 1 -- 172.21.15.62:0/3901166118 learned_addr learned my addr 172.21.15.62:0/3901166118 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f657dc65700 1 -- 172.21.15.62:0/3901166118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb93e0 msgr2=0x559100fb9cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f657dc65700 1 --2- 172.21.15.62:0/3901166118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb93e0 0x559100fb9cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f657dc65700 1 -- 172.21.15.62:0/3901166118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 msgr2=0x559100fba750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f657dc65700 1 --2- 172.21.15.62:0/3901166118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 0x559100fba750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f657dc65700 1 -- 172.21.15.62:0/3901166118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559100d8a350 con 0x559100fb8a80 2024-05-08T22:37:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f657dc65700 1 --2- 172.21.15.62:0/3901166118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 0x559100fb8ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f657400b580 tx=0x7f6574005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84001bfab65b98da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f6563fff700 1 -- 172.21.15.62:0/3901166118 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657409e910 con 0x559100fb8a80 2024-05-08T22:37:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f6563fff700 1 -- 172.21.15.62:0/3901166118 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65740a2ed0 con 0x559100fb8a80 2024-05-08T22:37:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.190+0000 7f6563fff700 1 -- 172.21.15.62:0/3901166118 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657409a1d0 con 0x559100fb8a80 2024-05-08T22:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.191+0000 7f658c083b80 1 -- 172.21.15.62:0/3901166118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 msgr2=0x559100fb8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.191+0000 7f658c083b80 1 --2- 172.21.15.62:0/3901166118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 0x559100fb8ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f657400b580 tx=0x7f6574005ef0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.191+0000 7f658c083b80 1 -- 172.21.15.62:0/3901166118 shutdown_connections 2024-05-08T22:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.191+0000 7f658c083b80 1 --2- 172.21.15.62:0/3901166118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 0x559100fba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.191+0000 7f658c083b80 1 --2- 172.21.15.62:0/3901166118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 0x559100fb8ea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.191+0000 7f658c083b80 1 --2- 172.21.15.62:0/3901166118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb93e0 0x559100fb9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.191+0000 7f658c083b80 1 -- 172.21.15.62:0/3901166118 >> 172.21.15.62:0/3901166118 conn(0x559100f171b0 msgr2=0x559100fb5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.192+0000 7f658c083b80 1 -- 172.21.15.62:0/3901166118 shutdown_connections 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.192+0000 7f658c083b80 1 -- 172.21.15.62:0/3901166118 wait complete. 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.192+0000 7f658c083b80 1 -- start start 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.192+0000 7f658c083b80 1 Processor -- start 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.192+0000 7f658c083b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 0x559100fc2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.192+0000 7f658c083b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 0x559100fc2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.192+0000 7f658c083b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fba2d0 0x559100fc9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.192+0000 7f658c083b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100ea5500 con 0x559100fb93e0 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.192+0000 7f658c083b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100ea51a0 con 0x559100fba2d0 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.192+0000 7f658c083b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100d8c640 con 0x559100fb8a80 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.192+0000 7f658c083b80 10 client.0 _ll_register_callbacks cb 0x559100eaead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657dc65700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 0x559100fc2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657e466700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fba2d0 0x559100fc9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657dc65700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 0x559100fc2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.62:54294/0 (socket says 172.21.15.62:54294) 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657dc65700 1 -- 172.21.15.62:0/2611508633 learned_addr learned my addr 172.21.15.62:0/2611508633 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657d464700 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 0x559100fc2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657dc65700 1 -- 172.21.15.62:0/2611508633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fba2d0 msgr2=0x559100fc9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657dc65700 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fba2d0 0x559100fc9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657dc65700 1 -- 172.21.15.62:0/2611508633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 msgr2=0x559100fc2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657dc65700 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 0x559100fc2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657dc65700 1 -- 172.21.15.62:0/2611508633 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65781a13f0 con 0x559100fb8a80 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657e466700 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fba2d0 0x559100fc9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f657dc65700 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 0x559100fc2510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6574009990 tx=0x7f6574009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7091a1fd0fff05d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f65617fa700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f65617fa700 1 -- 172.21.15.62:0/2611508633 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65740b3370 con 0x559100fb8a80 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f658c083b80 1 -- 172.21.15.62:0/2611508633 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559100e62210 con 0x559100fb8a80 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f65617fa700 1 -- 172.21.15.62:0/2611508633 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f65740a2a40 con 0x559100fb8a80 2024-05-08T22:37:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f65617fa700 1 -- 172.21.15.62:0/2611508633 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65740a2c00 con 0x559100fb8a80 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f658c083b80 10 client.25102 fetch_fsmap learned FSMap version 29 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.193+0000 7f658c083b80 1 -- 172.21.15.62:0/2611508633 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559100d36b00 con 0x559100fb8a80 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.194+0000 7f65617fa700 1 -- 172.21.15.62:0/2611508633 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f657407d020 con 0x559100fb8a80 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.194+0000 7f65617fa700 1 -- 172.21.15.62:0/2611508633 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f65740662e0 con 0x559100fb8a80 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.194+0000 7f658c083b80 10 client.25102 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.194+0000 7f658c083b80 10 client.25102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.194+0000 7f658c083b80 20 client.25102 populate_metadata read hostname 'smithi062' 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.194+0000 7f658c083b80 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x559100fcdf60 0x559100fd0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.194+0000 7f658c083b80 4 client.25102 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.194+0000 7f658c083b80 1 -- 172.21.15.62:0/2611508633 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559100f887b0 con 0x559100fcdf60 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.194+0000 7f657d464700 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x559100fcdf60 0x559100fd0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f65617fa700 10 client.25102 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f657d464700 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x559100fcdf60 0x559100fd0380 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1b8c92d6e69f86 server_cookie=3ee7a096362232d4 in_seq=0 out_seq=0 2024-05-08T22:37:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f65617fa700 1 -- 172.21.15.62:0/2611508633 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559100f887b0 con 0x559100fcdf60 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f65617fa700 10 client.25102 handle_command_reply: tid=0 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f658c083b80 1 client.25102 shutdown 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f658c083b80 1 -- 172.21.15.62:0/2611508633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 msgr2=0x559100fc2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f658c083b80 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 0x559100fc2510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6574009990 tx=0x7f6574009310 comp rx=0 tx=0).stop 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f658c083b80 1 -- 172.21.15.62:0/2611508633 shutdown_connections 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f658c083b80 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x559100fcdf60 0x559100fd0380 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f658c083b80 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fba2d0 0x559100fc9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f658c083b80 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 0x559100fc2510 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f65617fa700 0 client.25102 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f658c083b80 1 --2- 172.21.15.62:0/2611508633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 0x559100fc2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f658c083b80 1 -- 172.21.15.62:0/2611508633 >> 172.21.15.62:0/2611508633 conn(0x559100f171b0 msgr2=0x559100fb3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f658c083b80 1 -- 172.21.15.62:0/2611508633 shutdown_connections 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.195+0000 7f658c083b80 1 -- 172.21.15.62:0/2611508633 wait complete. 2024-05-08T22:37:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.196+0000 7f658c083b80 20 client.25102 trim_cache size 0 max 16384 2024-05-08T22:37:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f658c083b80 1 Processor -- start 2024-05-08T22:37:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f658c083b80 1 -- start start 2024-05-08T22:37:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f658c083b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 0x559100fd0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f658c083b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 0x559100fc9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f658c083b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fba2d0 0x559100fba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f658c083b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100fb97c0 con 0x559100fb93e0 2024-05-08T22:37:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f658c083b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100fd5550 con 0x559100fb8a80 2024-05-08T22:37:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f658c083b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100fc3700 con 0x559100fba2d0 2024-05-08T22:37:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f657d464700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 0x559100fc9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f657d464700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 0x559100fc9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36638/0 (socket says 172.21.15.62:36638) 2024-05-08T22:37:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f657d464700 1 -- 172.21.15.62:0/1115497178 learned_addr learned my addr 172.21.15.62:0/1115497178 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f657e466700 1 --2- 172.21.15.62:0/1115497178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fba2d0 0x559100fba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.211+0000 7f657dc65700 1 --2- 172.21.15.62:0/1115497178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 0x559100fd0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.212+0000 7f657d464700 1 -- 172.21.15.62:0/1115497178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fba2d0 msgr2=0x559100fba6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.212+0000 7f657d464700 1 --2- 172.21.15.62:0/1115497178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fba2d0 0x559100fba6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.212+0000 7f657d464700 1 -- 172.21.15.62:0/1115497178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 msgr2=0x559100fd0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.212+0000 7f657d464700 1 --2- 172.21.15.62:0/1115497178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 0x559100fd0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.212+0000 7f657d464700 1 -- 172.21.15.62:0/1115497178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559100ecf500 con 0x559100fb93e0 2024-05-08T22:37:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.212+0000 7f657d464700 1 --2- 172.21.15.62:0/1115497178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 0x559100fc9a50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x559100fd0c50 tx=0x7f6568014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c854385d0146bd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.212+0000 7f656cff9700 1 -- 172.21.15.62:0/1115497178 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568020030 con 0x559100fb93e0 2024-05-08T22:37:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.212+0000 7f656cff9700 1 -- 172.21.15.62:0/1115497178 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6568021040 con 0x559100fb93e0 2024-05-08T22:37:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.212+0000 7f656cff9700 1 -- 172.21.15.62:0/1115497178 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568013040 con 0x559100fb93e0 2024-05-08T22:37:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.213+0000 7f658c083b80 1 -- 172.21.15.62:0/1115497178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 msgr2=0x559100fc9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.213+0000 7f658c083b80 1 --2- 172.21.15.62:0/1115497178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 0x559100fc9a50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x559100fd0c50 tx=0x7f6568014000 comp rx=0 tx=0).stop 2024-05-08T22:37:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.213+0000 7f658c083b80 1 -- 172.21.15.62:0/1115497178 shutdown_connections 2024-05-08T22:37:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.213+0000 7f658c083b80 1 --2- 172.21.15.62:0/1115497178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fba2d0 0x559100fba6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.213+0000 7f658c083b80 1 --2- 172.21.15.62:0/1115497178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb8a80 0x559100fd0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.213+0000 7f658c083b80 1 --2- 172.21.15.62:0/1115497178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fb93e0 0x559100fc9a50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.213+0000 7f658c083b80 1 -- 172.21.15.62:0/1115497178 >> 172.21.15.62:0/1115497178 conn(0x559100f171b0 msgr2=0x559100fc0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.213+0000 7f658c083b80 1 -- 172.21.15.62:0/1115497178 shutdown_connections 2024-05-08T22:37:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.213+0000 7f658c083b80 1 -- 172.21.15.62:0/1115497178 wait complete. 2024-05-08T22:37:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.213+0000 7f658c083b80 1 -- start start 2024-05-08T22:37:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.213+0000 7f658c083b80 1 Processor -- start 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f658c083b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 0x559100fc6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f658c083b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb93e0 0x559100fc73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f658c083b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 0x559100fc8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f658c083b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100fc88c0 con 0x559100fba2d0 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f658c083b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100fc8a30 con 0x559100fb93e0 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f658c083b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100fc8bf0 con 0x559100fb8a80 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f658c083b80 10 client.0 _ll_register_callbacks cb 0x559100eaead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657e466700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 0x559100fc8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657e466700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 0x559100fc8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36644/0 (socket says 172.21.15.62:36644) 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657e466700 1 -- 172.21.15.62:0/2269264010 learned_addr learned my addr 172.21.15.62:0/2269264010 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657d464700 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb93e0 0x559100fc73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657dc65700 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 0x559100fc6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657d464700 1 -- 172.21.15.62:0/2269264010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 msgr2=0x559100fc6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657d464700 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 0x559100fc6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657d464700 1 -- 172.21.15.62:0/2269264010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 msgr2=0x559100fc8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657d464700 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 0x559100fc8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657d464700 1 -- 172.21.15.62:0/2269264010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6568019400 con 0x559100fb93e0 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657e466700 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 0x559100fc8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f657d464700 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb93e0 0x559100fc73c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x559100fc35b0 tx=0x7f656800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11b248245873dbdb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.214+0000 7f65617fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f65617fa700 1 -- 172.21.15.62:0/2269264010 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568020070 con 0x559100fb93e0 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f65617fa700 1 -- 172.21.15.62:0/2269264010 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6568028260 con 0x559100fb93e0 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f658c083b80 1 -- 172.21.15.62:0/2269264010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65740a2ed0 con 0x559100fb93e0 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f65617fa700 1 -- 172.21.15.62:0/2269264010 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6568013050 con 0x559100fb93e0 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f658c083b80 10 client.25145 fetch_fsmap learned FSMap version 29 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f658c083b80 1 -- 172.21.15.62:0/2269264010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559100d8ec70 con 0x559100fb93e0 2024-05-08T22:37:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f65617fa700 1 -- 172.21.15.62:0/2269264010 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f656803f020 con 0x559100fb93e0 2024-05-08T22:37:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f65617fa700 1 -- 172.21.15.62:0/2269264010 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f656803b930 con 0x559100fb93e0 2024-05-08T22:37:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f658c083b80 10 client.25145 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:37:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f658c083b80 10 client.25145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:37:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f658c083b80 20 client.25145 populate_metadata read hostname 'smithi062' 2024-05-08T22:37:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f658c083b80 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x5591010249c0 0x559101026de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f658c083b80 4 client.25145 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:37:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f658c083b80 1 -- 172.21.15.62:0/2269264010 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559100f01e60 con 0x5591010249c0 2024-05-08T22:37:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.215+0000 7f657dc65700 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x5591010249c0 0x559101026de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.216+0000 7f65617fa700 10 client.25145 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.216+0000 7f657dc65700 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x5591010249c0 0x559101026de0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d9853bc810a0b3 server_cookie=fe1166ef58b8267a in_seq=0 out_seq=0 2024-05-08T22:37:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f65617fa700 1 -- 172.21.15.62:0/2269264010 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559100f01e60 con 0x5591010249c0 2024-05-08T22:37:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f65617fa700 10 client.25145 handle_command_reply: tid=0 2024-05-08T22:37:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 1 client.25145 shutdown 2024-05-08T22:37:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 1 -- 172.21.15.62:0/2269264010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb93e0 msgr2=0x559100fc73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb93e0 0x559100fc73c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x559100fc35b0 tx=0x7f656800ecd0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 1 -- 172.21.15.62:0/2269264010 shutdown_connections 2024-05-08T22:37:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x5591010249c0 0x559101026de0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559100fba2d0 0x559100fc8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f65617fa700 0 client.25145 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559100fb8a80 0x559100fc6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 1 --2- 172.21.15.62:0/2269264010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559100fb93e0 0x559100fc73c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 1 -- 172.21.15.62:0/2269264010 >> 172.21.15.62:0/2269264010 conn(0x559100f171b0 msgr2=0x559100fb5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 1 -- 172.21.15.62:0/2269264010 shutdown_connections 2024-05-08T22:37:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 1 -- 172.21.15.62:0/2269264010 wait complete. 2024-05-08T22:37:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.219+0000 7f658c083b80 20 client.25145 trim_cache size 0 max 16384 2024-05-08T22:37:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-05-08T22:37:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-05-08T22:37:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-05-08T22:37:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-05-08T22:37:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 -- 172.21.15.62:0/2049337662 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65640a1110 msgr2=0x7f65640a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65640a1110 0x7f65640a35d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f656800acd0 tx=0x7f6568010040 comp rx=0 tx=0).stop 2024-05-08T22:37:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 -- 172.21.15.62:0/2049337662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65780ff520 msgr2=0x7f6578199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65780ff520 0x7f6578199080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6574003730 tx=0x7f657400f690 comp rx=0 tx=0).stop 2024-05-08T22:37:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 -- 172.21.15.62:0/2049337662 shutdown_connections 2024-05-08T22:37:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65781010b0 0x7f65781a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65780ff520 0x7f6578199080 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f65640a1110 0x7f65640a35d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 --2- 172.21.15.62:0/2049337662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6578100710 0x7f65781995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 -- 172.21.15.62:0/2049337662 >> 172.21.15.62:0/2049337662 conn(0x7f65780faab0 msgr2=0x7f6578103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 -- 172.21.15.62:0/2049337662 shutdown_connections 2024-05-08T22:37:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:11.220+0000 7f657fec9700 1 -- 172.21.15.62:0/2049337662 wait complete. 2024-05-08T22:37:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:11 smithi066 ceph-mon[46830]: pgmap v284: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:37:11.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:37:11.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:11 smithi079 ceph-mon[46738]: pgmap v284: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:11.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:37:11.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:37:11.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:11 smithi062 ceph-mon[42571]: pgmap v284: 97 pgs: 97 active+clean; 690 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:11.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:11.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:37:11.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:37:11.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:37:13.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:13 smithi066 ceph-mon[46830]: pgmap v285: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-08T22:37:13.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:13 smithi079 ceph-mon[46738]: pgmap v285: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-08T22:37:13.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:13 smithi062 ceph-mon[42571]: pgmap v285: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-08T22:37:15.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:15 smithi066 ceph-mon[46830]: pgmap v286: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:15.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:15 smithi079 ceph-mon[46738]: pgmap v286: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:15.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:15 smithi062 ceph-mon[42571]: pgmap v286: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:17.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:17 smithi066 ceph-mon[46830]: pgmap v287: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:17.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:17 smithi079 ceph-mon[46738]: pgmap v287: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:17.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:17 smithi062 ceph-mon[42571]: pgmap v287: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:19.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:19 smithi066 ceph-mon[46830]: pgmap v288: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:19.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:19 smithi079 ceph-mon[46738]: pgmap v288: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:19.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:19 smithi062 ceph-mon[42571]: pgmap v288: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:21.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:21 smithi066 ceph-mon[46830]: pgmap v289: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:21.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:21 smithi079 ceph-mon[46738]: pgmap v289: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:21.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:21 smithi062 ceph-mon[42571]: pgmap v289: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:23.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:23 smithi066 ceph-mon[46830]: pgmap v290: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:23.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:23 smithi079 ceph-mon[46738]: pgmap v290: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:23.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:23 smithi062 ceph-mon[42571]: pgmap v290: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.349+0000 7f469d4ca700 1 -- 172.21.15.62:0/2093245507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46980ffdb0 msgr2=0x7f46981001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.349+0000 7f469d4ca700 1 --2- 172.21.15.62:0/2093245507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46980ffdb0 0x7f46981001d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4688009990 tx=0x7f4688009ca0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.349+0000 7f469d4ca700 1 -- 172.21.15.62:0/2093245507 shutdown_connections 2024-05-08T22:37:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.349+0000 7f469d4ca700 1 --2- 172.21.15.62:0/2093245507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4698101940 0x7f46981081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.349+0000 7f469d4ca700 1 --2- 172.21.15.62:0/2093245507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46980ffdb0 0x7f46981001d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.349+0000 7f469d4ca700 1 --2- 172.21.15.62:0/2093245507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698100fa0 0x7f4698101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.349+0000 7f469d4ca700 1 -- 172.21.15.62:0/2093245507 >> 172.21.15.62:0/2093245507 conn(0x7f46980fb340 msgr2=0x7f46980fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.349+0000 7f469d4ca700 1 -- 172.21.15.62:0/2093245507 shutdown_connections 2024-05-08T22:37:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.349+0000 7f469d4ca700 1 -- 172.21.15.62:0/2093245507 wait complete. 2024-05-08T22:37:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.349+0000 7f469d4ca700 1 Processor -- start 2024-05-08T22:37:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f469d4ca700 1 -- start start 2024-05-08T22:37:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f469d4ca700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46980ffdb0 0x7f469810fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f469d4ca700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698100fa0 0x7f4698110410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f469d4ca700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4698101940 0x7f46981ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f469d4ca700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46981174c0 con 0x7f46980ffdb0 2024-05-08T22:37:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f469d4ca700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4698117600 con 0x7f4698100fa0 2024-05-08T22:37:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f469d4ca700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469806a9c0 con 0x7f4698101940 2024-05-08T22:37:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f46967fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698100fa0 0x7f4698110410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f46977fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4698101940 0x7f46981ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f46977fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4698101940 0x7f46981ad500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.62:46064/0 (socket says 172.21.15.62:46064) 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f46977fe700 1 -- 172.21.15.62:0/893269120 learned_addr learned my addr 172.21.15.62:0/893269120 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.350+0000 7f46967fc700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698100fa0 0x7f4698110410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.62:47530/0 (socket says 172.21.15.62:47530) 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.351+0000 7f46977fe700 1 -- 172.21.15.62:0/893269120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698100fa0 msgr2=0x7f4698110410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.351+0000 7f46977fe700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698100fa0 0x7f4698110410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.351+0000 7f46977fe700 1 -- 172.21.15.62:0/893269120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46980ffdb0 msgr2=0x7f469810fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.351+0000 7f46977fe700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46980ffdb0 0x7f469810fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.351+0000 7f46977fe700 1 -- 172.21.15.62:0/893269120 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880095f0 con 0x7f4698101940 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.351+0000 7f46977fe700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4698101940 0x7f46981ad500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f469000ef30 tx=0x7f469000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.351+0000 7f467ffff700 1 -- 172.21.15.62:0/893269120 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690018070 con 0x7f4698101940 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.351+0000 7f469d4ca700 1 -- 172.21.15.62:0/893269120 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469806abf0 con 0x7f4698101940 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.351+0000 7f467ffff700 1 -- 172.21.15.62:0/893269120 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f469000f040 con 0x7f4698101940 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.351+0000 7f467ffff700 1 -- 172.21.15.62:0/893269120 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469000cd90 con 0x7f4698101940 2024-05-08T22:37:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.351+0000 7f469d4ca700 1 -- 172.21.15.62:0/893269120 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469806b100 con 0x7f4698101940 2024-05-08T22:37:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.354+0000 7f467ffff700 1 -- 172.21.15.62:0/893269120 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4690004ad0 con 0x7f4698101940 2024-05-08T22:37:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.355+0000 7f467ffff700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46800a1190 0x7f46800a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.355+0000 7f467ffff700 1 -- 172.21.15.62:0/893269120 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f46900cc400 con 0x7f4698101940 2024-05-08T22:37:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f4696ffd700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46800a1190 0x7f46800a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46a9684b80 1 Processor -- start 2024-05-08T22:37:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46a9684b80 1 -- start start 2024-05-08T22:37:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d6dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d6ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d6f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46a9684b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9cc50f0 con 0x55c5e9d6da80 2024-05-08T22:37:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46a9684b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9cc2ab0 con 0x55c5e9d6e3e0 2024-05-08T22:37:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46a9684b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9cbe3a0 con 0x55c5e9d6f2d0 2024-05-08T22:37:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46967fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d6ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46967fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d6ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.62:47532/0 (socket says 172.21.15.62:47532) 2024-05-08T22:37:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46977fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d6f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46967fc700 1 -- 172.21.15.62:0/2718638940 learned_addr learned my addr 172.21.15.62:0/2718638940 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46967fc700 1 -- 172.21.15.62:0/2718638940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6f2d0 msgr2=0x55c5e9d6f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.356+0000 7f46967fc700 1 --2- 172.21.15.62:0/2718638940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d6f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.357+0000 7f46967fc700 1 -- 172.21.15.62:0/2718638940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 msgr2=0x55c5e9d6dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.357+0000 7f46967fc700 1 --2- 172.21.15.62:0/2718638940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d6dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.357+0000 7f46967fc700 1 -- 172.21.15.62:0/2718638940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5e9b3f350 con 0x55c5e9d6e3e0 2024-05-08T22:37:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.357+0000 7f46977fe700 1 --2- 172.21.15.62:0/2718638940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d6f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.357+0000 7f46967fc700 1 --2- 172.21.15.62:0/2718638940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d6ecf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f468c009d80 tx=0x7f468c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b38d0bcd93d25744 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.357+0000 7f467d7fa700 1 -- 172.21.15.62:0/2718638940 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c004230 con 0x55c5e9d6e3e0 2024-05-08T22:37:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.357+0000 7f467d7fa700 1 -- 172.21.15.62:0/2718638940 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f468c004390 con 0x55c5e9d6e3e0 2024-05-08T22:37:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.357+0000 7f467d7fa700 1 -- 172.21.15.62:0/2718638940 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c0197a0 con 0x55c5e9d6e3e0 2024-05-08T22:37:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.358+0000 7f46a9684b80 1 -- 172.21.15.62:0/2718638940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6e3e0 msgr2=0x55c5e9d6ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.358+0000 7f46a9684b80 1 --2- 172.21.15.62:0/2718638940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d6ecf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f468c009d80 tx=0x7f468c011720 comp rx=0 tx=0).stop 2024-05-08T22:37:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.358+0000 7f46a9684b80 1 -- 172.21.15.62:0/2718638940 shutdown_connections 2024-05-08T22:37:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.358+0000 7f46a9684b80 1 --2- 172.21.15.62:0/2718638940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d6f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.358+0000 7f46a9684b80 1 --2- 172.21.15.62:0/2718638940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d6dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.358+0000 7f46a9684b80 1 --2- 172.21.15.62:0/2718638940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d6ecf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.358+0000 7f46a9684b80 1 -- 172.21.15.62:0/2718638940 >> 172.21.15.62:0/2718638940 conn(0x55c5e9ccc1b0 msgr2=0x55c5e9d6a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f4696ffd700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46800a1190 0x7f46800a3650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f46880098e0 tx=0x7f4688005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f46a9684b80 1 -- 172.21.15.62:0/2718638940 shutdown_connections 2024-05-08T22:37:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f46a9684b80 1 -- 172.21.15.62:0/2718638940 wait complete. 2024-05-08T22:37:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f46a9684b80 1 -- start start 2024-05-08T22:37:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f46a9684b80 1 Processor -- start 2024-05-08T22:37:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d77800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d77d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d7ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f46a9684b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9c5a500 con 0x55c5e9d6da80 2024-05-08T22:37:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f46a9684b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9c5a1a0 con 0x55c5e9d6f2d0 2024-05-08T22:37:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f46a9684b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9b41640 con 0x55c5e9d6e3e0 2024-05-08T22:37:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.359+0000 7f46a9684b80 10 client.0 _ll_register_callbacks cb 0x55c5e9c63ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f4696ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d77800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f4696ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d77800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52858/0 (socket says 172.21.15.62:52858) 2024-05-08T22:37:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f4696ffd700 1 -- 172.21.15.62:0/1983947704 learned_addr learned my addr 172.21.15.62:0/1983947704 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f4696ffd700 1 -- 172.21.15.62:0/1983947704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6e3e0 msgr2=0x55c5e9d77d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f4696ffd700 1 --2- 172.21.15.62:0/1983947704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d77d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f4696ffd700 1 -- 172.21.15.62:0/1983947704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6f2d0 msgr2=0x55c5e9d7ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f4696ffd700 1 --2- 172.21.15.62:0/1983947704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d7ed80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f4696ffd700 1 -- 172.21.15.62:0/1983947704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469806b100 con 0x55c5e9d6da80 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f4696ffd700 1 --2- 172.21.15.62:0/1983947704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d77800 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4688009990 tx=0x7f4688009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e15bce7f1f405a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f467aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f467aff5700 1 -- 172.21.15.62:0/1983947704 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688004cf0 con 0x55c5e9d6da80 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.360+0000 7f467aff5700 1 -- 172.21.15.62:0/1983947704 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f468800f460 con 0x55c5e9d6da80 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f46a9684b80 1 -- 172.21.15.62:0/1983947704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5e9c17210 con 0x55c5e9d6da80 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f467aff5700 1 -- 172.21.15.62:0/1983947704 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4688004e50 con 0x55c5e9d6da80 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f46a9684b80 10 client.15447 fetch_fsmap learned FSMap version 29 2024-05-08T22:37:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f46a9684b80 1 -- 172.21.15.62:0/1983947704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f468c0099e0 con 0x55c5e9d6da80 2024-05-08T22:37:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f467aff5700 1 -- 172.21.15.62:0/1983947704 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5e9c17210 con 0x55c5e9d6da80 2024-05-08T22:37:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f467aff5700 1 -- 172.21.15.62:0/1983947704 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4688028720 con 0x55c5e9d6da80 2024-05-08T22:37:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f46a9684b80 10 client.15447 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:37:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f46a9684b80 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:37:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f46a9684b80 20 client.15447 populate_metadata read hostname 'smithi062' 2024-05-08T22:37:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f46a9684b80 1 --2- 172.21.15.62:0/1983947704 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55c5e9d83690 0x55c5e9d85ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f46a9684b80 4 client.15447 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:37:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.361+0000 7f46a9684b80 1 -- 172.21.15.62:0/1983947704 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5e9d3d7b0 con 0x55c5e9d83690 2024-05-08T22:37:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.362+0000 7f46967fc700 1 --2- 172.21.15.62:0/1983947704 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55c5e9d83690 0x55c5e9d85ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.362+0000 7f467aff5700 10 client.15447 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.362+0000 7f46967fc700 1 --2- 172.21.15.62:0/1983947704 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55c5e9d83690 0x55c5e9d85ab0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40edb58e689f6e3e server_cookie=870982d8e6b733c9 in_seq=0 out_seq=0 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.363+0000 7f467aff5700 1 -- 172.21.15.62:0/1983947704 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5e9d3d7b0 con 0x55c5e9d83690 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.363+0000 7f467aff5700 10 client.15447 handle_command_reply: tid=0 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.363+0000 7f46a9684b80 1 client.15447 shutdown 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.363+0000 7f46a9684b80 1 -- 172.21.15.62:0/1983947704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 msgr2=0x55c5e9d77800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.363+0000 7f46a9684b80 1 --2- 172.21.15.62:0/1983947704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d77800 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4688009990 tx=0x7f4688009310 comp rx=0 tx=0).stop 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.365+0000 7f46a9684b80 1 -- 172.21.15.62:0/1983947704 shutdown_connections 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.365+0000 7f46a9684b80 1 --2- 172.21.15.62:0/1983947704 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55c5e9d83690 0x55c5e9d85ab0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.365+0000 7f46a9684b80 1 --2- 172.21.15.62:0/1983947704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d7ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.365+0000 7f46a9684b80 1 --2- 172.21.15.62:0/1983947704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d77800 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.365+0000 7f467aff5700 0 client.15447 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.365+0000 7f46a9684b80 1 --2- 172.21.15.62:0/1983947704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d77d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.365+0000 7f46a9684b80 1 -- 172.21.15.62:0/1983947704 >> 172.21.15.62:0/1983947704 conn(0x55c5e9ccc1b0 msgr2=0x55c5e9d68860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.366+0000 7f46a9684b80 1 -- 172.21.15.62:0/1983947704 shutdown_connections 2024-05-08T22:37:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.366+0000 7f46a9684b80 1 -- 172.21.15.62:0/1983947704 wait complete. 2024-05-08T22:37:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.366+0000 7f46a9684b80 20 client.15447 trim_cache size 0 max 16384 2024-05-08T22:37:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.381+0000 7f46a9684b80 1 Processor -- start 2024-05-08T22:37:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.381+0000 7f46a9684b80 1 -- start start 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.381+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.381+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d7ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.381+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d76d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.381+0000 7f46a9684b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9d69c40 con 0x55c5e9d6da80 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.381+0000 7f46a9684b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9d72db0 con 0x55c5e9d6f2d0 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.381+0000 7f46a9684b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9d69930 con 0x55c5e9d6e3e0 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.381+0000 7f4696ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.381+0000 7f4696ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52866/0 (socket says 172.21.15.62:52866) 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.381+0000 7f4696ffd700 1 -- 172.21.15.62:0/1214153927 learned_addr learned my addr 172.21.15.62:0/1214153927 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.382+0000 7f4696ffd700 1 -- 172.21.15.62:0/1214153927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6e3e0 msgr2=0x55c5e9d7ed40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.382+0000 7f4696ffd700 1 --2- 172.21.15.62:0/1214153927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d7ed40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.382+0000 7f4696ffd700 1 -- 172.21.15.62:0/1214153927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6f2d0 msgr2=0x55c5e9d76d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.382+0000 7f4696ffd700 1 --2- 172.21.15.62:0/1214153927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d76d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.382+0000 7f4696ffd700 1 -- 172.21.15.62:0/1214153927 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5e9aebb00 con 0x55c5e9d6da80 2024-05-08T22:37:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.382+0000 7f4696ffd700 1 --2- 172.21.15.62:0/1214153927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7f430 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4688036040 tx=0x7f4688004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7767392156146ae8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.383+0000 7f467dffb700 1 -- 172.21.15.62:0/1214153927 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688004df0 con 0x55c5e9d6da80 2024-05-08T22:37:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.383+0000 7f467dffb700 1 -- 172.21.15.62:0/1214153927 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46880288c0 con 0x55c5e9d6da80 2024-05-08T22:37:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.383+0000 7f467dffb700 1 -- 172.21.15.62:0/1214153927 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468803eb50 con 0x55c5e9d6da80 2024-05-08T22:37:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.383+0000 7f46a9684b80 1 -- 172.21.15.62:0/1214153927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 msgr2=0x55c5e9d7f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.383+0000 7f46a9684b80 1 --2- 172.21.15.62:0/1214153927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7f430 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4688036040 tx=0x7f4688004960 comp rx=0 tx=0).stop 2024-05-08T22:37:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 -- 172.21.15.62:0/1214153927 shutdown_connections 2024-05-08T22:37:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 --2- 172.21.15.62:0/1214153927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d76d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 --2- 172.21.15.62:0/1214153927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7f430 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 --2- 172.21.15.62:0/1214153927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d7ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 -- 172.21.15.62:0/1214153927 >> 172.21.15.62:0/1214153927 conn(0x55c5e9ccc1b0 msgr2=0x55c5e9d76540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 -- 172.21.15.62:0/1214153927 shutdown_connections 2024-05-08T22:37:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 -- 172.21.15.62:0/1214153927 wait complete. 2024-05-08T22:37:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 -- start start 2024-05-08T22:37:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 Processor -- start 2024-05-08T22:37:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d7c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d7d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9d7df10 con 0x55c5e9d6da80 2024-05-08T22:37:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9d7e080 con 0x55c5e9d6e3e0 2024-05-08T22:37:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.384+0000 7f46a9684b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e9d7e240 con 0x55c5e9d6f2d0 2024-05-08T22:37:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f46a9684b80 10 client.0 _ll_register_callbacks cb 0x55c5e9c63ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f4696ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f4696ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52874/0 (socket says 172.21.15.62:52874) 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f4696ffd700 1 -- 172.21.15.62:0/3589664780 learned_addr learned my addr 172.21.15.62:0/3589664780 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f46977fe700 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d7d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f4696ffd700 1 -- 172.21.15.62:0/3589664780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6f2d0 msgr2=0x55c5e9d7d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f4696ffd700 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d7d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f4696ffd700 1 -- 172.21.15.62:0/3589664780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6e3e0 msgr2=0x55c5e9d7c9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f4696ffd700 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d7c9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f4696ffd700 1 -- 172.21.15.62:0/3589664780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c0099e0 con 0x55c5e9d6da80 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f4696ffd700 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7c480 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4688028b00 tx=0x7f4688004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04a4586b70f5285 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.385+0000 7f467aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f467aff5700 1 -- 172.21.15.62:0/3589664780 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46880286d0 con 0x55c5e9d6da80 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f467aff5700 1 -- 172.21.15.62:0/3589664780 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f468803e850 con 0x55c5e9d6da80 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f46a9684b80 1 -- 172.21.15.62:0/3589664780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f468c004390 con 0x55c5e9d6da80 2024-05-08T22:37:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f467aff5700 1 -- 172.21.15.62:0/3589664780 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f468803ea10 con 0x55c5e9d6da80 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f46a9684b80 10 client.15453 fetch_fsmap learned FSMap version 29 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f46a9684b80 1 -- 172.21.15.62:0/3589664780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c5e9c84500 con 0x55c5e9d6da80 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f467aff5700 1 -- 172.21.15.62:0/3589664780 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f468c004390 con 0x55c5e9d6da80 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f467aff5700 1 -- 172.21.15.62:0/3589664780 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4688034bd0 con 0x55c5e9d6da80 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f46a9684b80 10 client.15453 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f46a9684b80 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f46a9684b80 20 client.15453 populate_metadata read hostname 'smithi062' 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f46a9684b80 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55c5e9dda0c0 0x55c5e9ddc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f46a9684b80 4 client.15453 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.386+0000 7f46a9684b80 1 -- 172.21.15.62:0/3589664780 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c5e9cb6e60 con 0x55c5e9dda0c0 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.387+0000 7f46967fc700 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55c5e9dda0c0 0x55c5e9ddc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.387+0000 7f46967fc700 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55c5e9dda0c0 0x55c5e9ddc4e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7b1ac1e647c15d server_cookie=3ca47affb27d4f81 in_seq=0 out_seq=0 2024-05-08T22:37:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.387+0000 7f467aff5700 10 client.15453 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f467aff5700 1 -- 172.21.15.62:0/3589664780 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c5e9cb6e60 con 0x55c5e9dda0c0 2024-05-08T22:37:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f467aff5700 10 client.15453 handle_command_reply: tid=0 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f46a9684b80 1 client.15453 shutdown 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f46a9684b80 1 -- 172.21.15.62:0/3589664780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 msgr2=0x55c5e9d7c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f46a9684b80 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7c480 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4688028b00 tx=0x7f4688004ed0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f46a9684b80 1 -- 172.21.15.62:0/3589664780 shutdown_connections 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f46a9684b80 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55c5e9dda0c0 0x55c5e9ddc4e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f46a9684b80 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c5e9d6f2d0 0x55c5e9d7d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f46a9684b80 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5e9d6da80 0x55c5e9d7c480 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f467aff5700 0 client.15453 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f46a9684b80 1 --2- 172.21.15.62:0/3589664780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5e9d6e3e0 0x55c5e9d7c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f46a9684b80 1 -- 172.21.15.62:0/3589664780 >> 172.21.15.62:0/3589664780 conn(0x55c5e9ccc1b0 msgr2=0x55c5e9d76540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f46a9684b80 1 -- 172.21.15.62:0/3589664780 shutdown_connections 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.391+0000 7f46a9684b80 1 -- 172.21.15.62:0/3589664780 wait complete. 2024-05-08T22:37:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f46a9684b80 20 client.15453 trim_cache size 0 max 16384 2024-05-08T22:37:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-05-08T22:37:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-05-08T22:37:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-05-08T22:37:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-05-08T22:37:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f469d4ca700 1 -- 172.21.15.62:0/893269120 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46800a1190 msgr2=0x7f46800a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f469d4ca700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46800a1190 0x7f46800a3650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f46880098e0 tx=0x7f4688005be0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f469d4ca700 1 -- 172.21.15.62:0/893269120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4698101940 msgr2=0x7f46981ad500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f469d4ca700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4698101940 0x7f46981ad500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f469000ef30 tx=0x7f469000c3a0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f469d4ca700 1 -- 172.21.15.62:0/893269120 shutdown_connections 2024-05-08T22:37:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f469d4ca700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4698101940 0x7f46981ad500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f469d4ca700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46980ffdb0 0x7f469810fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f469d4ca700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f46800a1190 0x7f46800a3650 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f469d4ca700 1 --2- 172.21.15.62:0/893269120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698100fa0 0x7f4698110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f469d4ca700 1 -- 172.21.15.62:0/893269120 >> 172.21.15.62:0/893269120 conn(0x7f46980fb340 msgr2=0x7f4698104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.392+0000 7f469d4ca700 1 -- 172.21.15.62:0/893269120 shutdown_connections 2024-05-08T22:37:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:24.393+0000 7f469d4ca700 1 -- 172.21.15.62:0/893269120 wait complete. 2024-05-08T22:37:25.824 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:25 smithi066 ceph-mon[46830]: pgmap v291: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-08T22:37:25.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:25 smithi079 ceph-mon[46738]: pgmap v291: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-08T22:37:25.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:25 smithi062 ceph-mon[42571]: pgmap v291: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-08T22:37:27.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:27 smithi066 ceph-mon[46830]: pgmap v292: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-08T22:37:27.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:27 smithi079 ceph-mon[46738]: pgmap v292: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-08T22:37:27.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:27 smithi062 ceph-mon[42571]: pgmap v292: 97 pgs: 97 active+clean; 695 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-08T22:37:29.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:29 smithi066 ceph-mon[46830]: pgmap v293: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:29.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:29 smithi079 ceph-mon[46738]: pgmap v293: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:29.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:29 smithi062 ceph-mon[42571]: pgmap v293: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:31.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:31 smithi066 ceph-mon[46830]: pgmap v294: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:31.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:31 smithi079 ceph-mon[46738]: pgmap v294: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:31.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:31 smithi062 ceph-mon[42571]: pgmap v294: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:33.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:33 smithi066 ceph-mon[46830]: pgmap v295: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:33.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:33 smithi079 ceph-mon[46738]: pgmap v295: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:33.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:33 smithi062 ceph-mon[42571]: pgmap v295: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:35.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:35 smithi066 ceph-mon[46830]: pgmap v296: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:35.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:35 smithi079 ceph-mon[46738]: pgmap v296: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:35.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:35 smithi062 ceph-mon[42571]: pgmap v296: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.563+0000 7f817a486700 1 -- 172.21.15.62:0/3391717728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81740ffda0 msgr2=0x7f81741001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.563+0000 7f817a486700 1 --2- 172.21.15.62:0/3391717728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81740ffda0 0x7f81741001c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81640098f0 tx=0x7f8164009c00 comp rx=0 tx=0).stop 2024-05-08T22:37:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.564+0000 7f817a486700 1 -- 172.21.15.62:0/3391717728 shutdown_connections 2024-05-08T22:37:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.564+0000 7f817a486700 1 --2- 172.21.15.62:0/3391717728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8174101930 0x7f81741081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.564+0000 7f817a486700 1 --2- 172.21.15.62:0/3391717728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81740ffda0 0x7f81741001c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.564+0000 7f817a486700 1 --2- 172.21.15.62:0/3391717728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8174100f90 0x7f81741013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.564+0000 7f817a486700 1 -- 172.21.15.62:0/3391717728 >> 172.21.15.62:0/3391717728 conn(0x7f81740fb310 msgr2=0x7f81740fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.564+0000 7f817a486700 1 -- 172.21.15.62:0/3391717728 shutdown_connections 2024-05-08T22:37:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.564+0000 7f817a486700 1 -- 172.21.15.62:0/3391717728 wait complete. 2024-05-08T22:37:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.564+0000 7f817a486700 1 Processor -- start 2024-05-08T22:37:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f817a486700 1 -- start start 2024-05-08T22:37:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f817a486700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81740ffda0 0x7f8174196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f817a486700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8174100f90 0x7f81741973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f817a486700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8174101930 0x7f817419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f817a486700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817419e9d0 con 0x7f8174100f90 2024-05-08T22:37:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f817a486700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817406a9c0 con 0x7f81740ffda0 2024-05-08T22:37:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f817a486700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817406ab50 con 0x7f8174101930 2024-05-08T22:37:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f81737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8174100f90 0x7f81741973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f81737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8174100f90 0x7f81741973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45042/0 (socket says 172.21.15.62:45042) 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f81737fe700 1 -- 172.21.15.62:0/748156525 learned_addr learned my addr 172.21.15.62:0/748156525 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f8173fff700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81740ffda0 0x7f8174196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f8178a23700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8174101930 0x7f817419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f81737fe700 1 -- 172.21.15.62:0/748156525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8174101930 msgr2=0x7f817419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f81737fe700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8174101930 0x7f817419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f81737fe700 1 -- 172.21.15.62:0/748156525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81740ffda0 msgr2=0x7f8174196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.565+0000 7f81737fe700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81740ffda0 0x7f8174196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.566+0000 7f81737fe700 1 -- 172.21.15.62:0/748156525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c009d00 con 0x7f8174100f90 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.566+0000 7f81737fe700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8174100f90 0x7f81741973e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f816800e9a0 tx=0x7f816800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.566+0000 7f81717fa700 1 -- 172.21.15.62:0/748156525 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168009790 con 0x7f8174100f90 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.566+0000 7f81717fa700 1 -- 172.21.15.62:0/748156525 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f816800cbd0 con 0x7f8174100f90 2024-05-08T22:37:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.566+0000 7f81717fa700 1 -- 172.21.15.62:0/748156525 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168018680 con 0x7f8174100f90 2024-05-08T22:37:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.566+0000 7f817a486700 1 -- 172.21.15.62:0/748156525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8164009520 con 0x7f8174100f90 2024-05-08T22:37:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.566+0000 7f817a486700 1 -- 172.21.15.62:0/748156525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817406b160 con 0x7f8174100f90 2024-05-08T22:37:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.570+0000 7f81717fa700 1 -- 172.21.15.62:0/748156525 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f816800cd40 con 0x7f8174100f90 2024-05-08T22:37:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.572+0000 7f8186640b80 1 Processor -- start 2024-05-08T22:37:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.572+0000 7f8186640b80 1 -- start start 2024-05-08T22:37:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.572+0000 7f8186640b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247e9b0 0x55899247edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.572+0000 7f8186640b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x55899247fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.572+0000 7f8178a23700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247e9b0 0x55899247edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8178a23700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247e9b0 0x55899247edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45054/0 (socket says 172.21.15.62:45054) 2024-05-08T22:37:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8186640b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558992480200 0x558992480680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f81717fa700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f815c0a1210 0x7f815c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8186640b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589923d60f0 con 0x55899247e9b0 2024-05-08T22:37:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8186640b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589923d3ab0 con 0x558992480200 2024-05-08T22:37:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8186640b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589923cf3a0 con 0x55899247f310 2024-05-08T22:37:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8178a23700 1 -- 172.21.15.62:0/1594632575 learned_addr learned my addr 172.21.15.62:0/1594632575 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8173fff700 1 --2- 172.21.15.62:0/1594632575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x55899247fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8178a23700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f815c0a1210 0x7f815c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f81737fe700 1 --2- 172.21.15.62:0/1594632575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558992480200 0x558992480680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f81717fa700 1 -- 172.21.15.62:0/748156525 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8168014070 con 0x7f8174100f90 2024-05-08T22:37:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8173fff700 1 -- 172.21.15.62:0/1594632575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558992480200 msgr2=0x558992480680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8173fff700 1 --2- 172.21.15.62:0/1594632575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558992480200 0x558992480680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8173fff700 1 -- 172.21.15.62:0/1594632575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247e9b0 msgr2=0x55899247edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8173fff700 1 --2- 172.21.15.62:0/1594632575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247e9b0 0x55899247edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.573+0000 7f8173fff700 1 -- 172.21.15.62:0/1594632575 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558992250350 con 0x55899247f310 2024-05-08T22:37:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.574+0000 7f8173fff700 1 --2- 172.21.15.62:0/1594632575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x55899247fc70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8164009230 tx=0x7f8164009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51d9c66b197d767c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.574+0000 7f815a7fc700 1 -- 172.21.15.62:0/1594632575 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816401c040 con 0x55899247f310 2024-05-08T22:37:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.574+0000 7f815a7fc700 1 -- 172.21.15.62:0/1594632575 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f816400ba20 con 0x55899247f310 2024-05-08T22:37:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.574+0000 7f815a7fc700 1 -- 172.21.15.62:0/1594632575 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640054a0 con 0x55899247f310 2024-05-08T22:37:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.574+0000 7f8178a23700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f815c0a1210 0x7f815c0a36d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f816c009fd0 tx=0x7f816c0091c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.575+0000 7f8186640b80 1 -- 172.21.15.62:0/1594632575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 msgr2=0x55899247fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.575+0000 7f8186640b80 1 --2- 172.21.15.62:0/1594632575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x55899247fc70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8164009230 tx=0x7f8164009400 comp rx=0 tx=0).stop 2024-05-08T22:37:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.575+0000 7f8186640b80 1 -- 172.21.15.62:0/1594632575 shutdown_connections 2024-05-08T22:37:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.575+0000 7f8186640b80 1 --2- 172.21.15.62:0/1594632575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558992480200 0x558992480680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.575+0000 7f8186640b80 1 --2- 172.21.15.62:0/1594632575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247e9b0 0x55899247edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.575+0000 7f8186640b80 1 --2- 172.21.15.62:0/1594632575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x55899247fc70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.575+0000 7f8186640b80 1 -- 172.21.15.62:0/1594632575 >> 172.21.15.62:0/1594632575 conn(0x5589923dd1b0 msgr2=0x55899247b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.575+0000 7f8186640b80 1 -- 172.21.15.62:0/1594632575 shutdown_connections 2024-05-08T22:37:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.575+0000 7f8186640b80 1 -- 172.21.15.62:0/1594632575 wait complete. 2024-05-08T22:37:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.576+0000 7f8186640b80 1 -- start start 2024-05-08T22:37:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.576+0000 7f8186640b80 1 Processor -- start 2024-05-08T22:37:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.576+0000 7f8186640b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247e9b0 0x558992491080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.576+0000 7f8186640b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x5589924915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.576+0000 7f8186640b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558992480200 0x558992498600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.576+0000 7f8186640b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899236b500 con 0x55899247e9b0 2024-05-08T22:37:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.576+0000 7f8186640b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899236b1a0 con 0x558992480200 2024-05-08T22:37:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.576+0000 7f8186640b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992252640 con 0x55899247f310 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.577+0000 7f81737fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x5589924915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.577+0000 7f81737fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x5589924915c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.62:40576/0 (socket says 172.21.15.62:40576) 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.577+0000 7f81737fe700 1 -- 172.21.15.62:0/2800129590 learned_addr learned my addr 172.21.15.62:0/2800129590 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.577+0000 7f81737fe700 1 -- 172.21.15.62:0/2800129590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558992480200 msgr2=0x558992498600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.577+0000 7f81737fe700 1 --2- 172.21.15.62:0/2800129590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558992480200 0x558992498600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.577+0000 7f81737fe700 1 -- 172.21.15.62:0/2800129590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247e9b0 msgr2=0x558992491080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.577+0000 7f81737fe700 1 --2- 172.21.15.62:0/2800129590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247e9b0 0x558992491080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.577+0000 7f81737fe700 1 -- 172.21.15.62:0/2800129590 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81740685b0 con 0x55899247f310 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.577+0000 7f81737fe700 1 --2- 172.21.15.62:0/2800129590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x5589924915c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8168000c00 tx=0x7f8168099ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e14640221123a90a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.577+0000 7f81537fe700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.577+0000 7f81537fe700 1 -- 172.21.15.62:0/2800129590 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168099710 con 0x55899247f310 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.578+0000 7f81537fe700 1 -- 172.21.15.62:0/2800129590 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f816809de50 con 0x55899247f310 2024-05-08T22:37:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.578+0000 7f81537fe700 10 client.0 inactive, discarding osd_map(81..81 src has 1..81) v4 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.578+0000 7f8186640b80 10 client.0 _ll_register_callbacks cb 0x558992374ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.578+0000 7f8186640b80 1 -- 172.21.15.62:0/2800129590 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558992328210 con 0x55899247f310 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.578+0000 7f81537fe700 1 -- 172.21.15.62:0/2800129590 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81680a24a0 con 0x55899247f310 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.578+0000 7f8186640b80 10 client.25132 fetch_fsmap learned FSMap version 29 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.578+0000 7f8186640b80 1 -- 172.21.15.62:0/2800129590 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x5589921fcb00 con 0x55899247f310 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.579+0000 7f81537fe700 1 -- 172.21.15.62:0/2800129590 <== mon.1 v2:172.21.15.79:3300/0 4 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f81680b2bd0 con 0x55899247f310 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.579+0000 7f8186640b80 10 client.25132 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.579+0000 7f8186640b80 10 client.25132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.579+0000 7f8186640b80 20 client.25132 populate_metadata read hostname 'smithi062' 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.579+0000 7f8186640b80 1 --2- 172.21.15.62:0/2800129590 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55899249ccb0 0x55899249f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.579+0000 7f8186640b80 4 client.25132 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.579+0000 7f8186640b80 1 -- 172.21.15.62:0/2800129590 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55899244e7b0 con 0x55899249ccb0 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.579+0000 7f8173fff700 1 --2- 172.21.15.62:0/2800129590 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55899249ccb0 0x55899249f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.580+0000 7f81537fe700 10 client.25132 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.580+0000 7f8173fff700 1 --2- 172.21.15.62:0/2800129590 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55899249ccb0 0x55899249f170 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ae3712e88d5eb5 server_cookie=259a5124d621cbeb in_seq=0 out_seq=0 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.580+0000 7f81537fe700 1 -- 172.21.15.62:0/2800129590 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55899244e7b0 con 0x55899249ccb0 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.580+0000 7f81537fe700 10 client.25132 handle_command_reply: tid=0 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.580+0000 7f8186640b80 1 client.25132 shutdown 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f8186640b80 1 -- 172.21.15.62:0/2800129590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 msgr2=0x5589924915c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f8186640b80 1 --2- 172.21.15.62:0/2800129590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x5589924915c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8168000c00 tx=0x7f8168099ab0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f8186640b80 1 -- 172.21.15.62:0/2800129590 shutdown_connections 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f8186640b80 1 --2- 172.21.15.62:0/2800129590 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x55899249ccb0 0x55899249f170 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f8186640b80 1 --2- 172.21.15.62:0/2800129590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558992480200 0x558992498600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f8186640b80 1 --2- 172.21.15.62:0/2800129590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247e9b0 0x558992491080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f81537fe700 0 client.25132 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f8186640b80 1 --2- 172.21.15.62:0/2800129590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x5589924915c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f8186640b80 1 -- 172.21.15.62:0/2800129590 >> 172.21.15.62:0/2800129590 conn(0x5589923dd1b0 msgr2=0x558992479860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f8186640b80 1 -- 172.21.15.62:0/2800129590 shutdown_connections 2024-05-08T22:37:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f8186640b80 1 -- 172.21.15.62:0/2800129590 wait complete. 2024-05-08T22:37:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.581+0000 7f8186640b80 20 client.25132 trim_cache size 0 max 16384 2024-05-08T22:37:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.609+0000 7f8186640b80 1 Processor -- start 2024-05-08T22:37:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.609+0000 7f8186640b80 1 -- start start 2024-05-08T22:37:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.610+0000 7f8186640b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55899247e9b0 0x558992498cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.610+0000 7f8186640b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247f310 0x5589924985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.610+0000 7f8186640b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558992480200 0x558992482480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.610+0000 7f8186640b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589924769b0 con 0x55899247f310 2024-05-08T22:37:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.610+0000 7f8186640b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992491f80 con 0x55899247e9b0 2024-05-08T22:37:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.610+0000 7f8186640b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992499780 con 0x558992480200 2024-05-08T22:37:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.610+0000 7f81737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247f310 0x5589924985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.610+0000 7f81737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247f310 0x5589924985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45066/0 (socket says 172.21.15.62:45066) 2024-05-08T22:37:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.610+0000 7f81737fe700 1 -- 172.21.15.62:0/3662998214 learned_addr learned my addr 172.21.15.62:0/3662998214 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.610+0000 7f8178a23700 1 --2- 172.21.15.62:0/3662998214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558992480200 0x558992482480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.611+0000 7f8178a23700 1 -- 172.21.15.62:0/3662998214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55899247e9b0 msgr2=0x558992498cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.611+0000 7f8178a23700 1 --2- 172.21.15.62:0/3662998214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55899247e9b0 0x558992498cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.611+0000 7f8178a23700 1 -- 172.21.15.62:0/3662998214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247f310 msgr2=0x5589924985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.611+0000 7f8178a23700 1 --2- 172.21.15.62:0/3662998214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247f310 0x5589924985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.611+0000 7f8178a23700 1 -- 172.21.15.62:0/3662998214 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558992395500 con 0x558992480200 2024-05-08T22:37:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.611+0000 7f8178a23700 1 --2- 172.21.15.62:0/3662998214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558992480200 0x558992482480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f816c019e30 tx=0x7f816c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=107fdd32b3271aac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.611+0000 7f815affd700 1 -- 172.21.15.62:0/3662998214 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c00e070 con 0x558992480200 2024-05-08T22:37:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.611+0000 7f815affd700 1 -- 172.21.15.62:0/3662998214 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f816c00f040 con 0x558992480200 2024-05-08T22:37:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.611+0000 7f815affd700 1 -- 172.21.15.62:0/3662998214 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c003dc0 con 0x558992480200 2024-05-08T22:37:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.612+0000 7f8186640b80 1 -- 172.21.15.62:0/3662998214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558992480200 msgr2=0x558992482480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.612+0000 7f8186640b80 1 --2- 172.21.15.62:0/3662998214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558992480200 0x558992482480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f816c019e30 tx=0x7f816c012000 comp rx=0 tx=0).stop 2024-05-08T22:37:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.612+0000 7f8186640b80 1 -- 172.21.15.62:0/3662998214 shutdown_connections 2024-05-08T22:37:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.612+0000 7f8186640b80 1 --2- 172.21.15.62:0/3662998214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558992480200 0x558992482480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.612+0000 7f8186640b80 1 --2- 172.21.15.62:0/3662998214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55899247e9b0 0x558992498cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.612+0000 7f8186640b80 1 --2- 172.21.15.62:0/3662998214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55899247f310 0x5589924985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.612+0000 7f8186640b80 1 -- 172.21.15.62:0/3662998214 >> 172.21.15.62:0/3662998214 conn(0x5589923dd1b0 msgr2=0x558992479860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.612+0000 7f8186640b80 1 -- 172.21.15.62:0/3662998214 shutdown_connections 2024-05-08T22:37:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.613+0000 7f8186640b80 1 -- 172.21.15.62:0/3662998214 wait complete. 2024-05-08T22:37:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.613+0000 7f8186640b80 1 -- start start 2024-05-08T22:37:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.613+0000 7f8186640b80 1 Processor -- start 2024-05-08T22:37:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.613+0000 7f8186640b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55899247e9b0 0x5589924933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.613+0000 7f8186640b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x558992493930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.613+0000 7f8186640b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558992480200 0x558992484a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.613+0000 7f8186640b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992493eb0 con 0x558992480200 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.613+0000 7f8186640b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992493ff0 con 0x55899247e9b0 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.613+0000 7f8186640b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992484fd0 con 0x55899247f310 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f8186640b80 10 client.0 _ll_register_callbacks cb 0x558992374ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f8173fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55899247e9b0 0x5589924933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f8173fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55899247e9b0 0x5589924933f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.62:38324/0 (socket says 172.21.15.62:38324) 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f8173fff700 1 -- 172.21.15.62:0/2910315891 learned_addr learned my addr 172.21.15.62:0/2910315891 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f81737fe700 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x558992493930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f8178a23700 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558992480200 0x558992484a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f8173fff700 1 -- 172.21.15.62:0/2910315891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 msgr2=0x558992493930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f8173fff700 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x558992493930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f8173fff700 1 -- 172.21.15.62:0/2910315891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558992480200 msgr2=0x558992484a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f8173fff700 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558992480200 0x558992484a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f8173fff700 1 -- 172.21.15.62:0/2910315891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816c019bf0 con 0x55899247e9b0 2024-05-08T22:37:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f8173fff700 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55899247e9b0 0x5589924933f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8164000c00 tx=0x7f8164009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d67dc4795be86420 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f81537fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.614+0000 7f81537fe700 1 -- 172.21.15.62:0/2910315891 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816401c040 con 0x55899247e9b0 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f81537fe700 1 -- 172.21.15.62:0/2910315891 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f816400b840 con 0x55899247e9b0 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f8186640b80 1 -- 172.21.15.62:0/2910315891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f816400ba20 con 0x55899247e9b0 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f81537fe700 1 -- 172.21.15.62:0/2910315891 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f816400f870 con 0x55899247e9b0 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f8186640b80 10 client.25166 fetch_fsmap learned FSMap version 29 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f8186640b80 1 -- 172.21.15.62:0/2910315891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558992254c70 con 0x55899247e9b0 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f81537fe700 1 -- 172.21.15.62:0/2910315891 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f816400ba20 con 0x55899247e9b0 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f81537fe700 1 -- 172.21.15.62:0/2910315891 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8164039070 con 0x55899247e9b0 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f8186640b80 10 client.25166 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f8186640b80 10 client.25166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f8186640b80 20 client.25166 populate_metadata read hostname 'smithi062' 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f8186640b80 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x5589924865d0 0x5589924889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f8186640b80 4 client.25166 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:37:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.615+0000 7f8186640b80 1 -- 172.21.15.62:0/2910315891 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5589923c7e60 con 0x5589924865d0 2024-05-08T22:37:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.616+0000 7f81737fe700 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x5589924865d0 0x5589924889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.616+0000 7f81537fe700 10 client.25166 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.616+0000 7f81737fe700 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x5589924865d0 0x5589924889f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7736f1519695db server_cookie=7c8c2d093b92a93f in_seq=0 out_seq=0 2024-05-08T22:37:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.620+0000 7f81537fe700 1 -- 172.21.15.62:0/2910315891 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5589923c7e60 con 0x5589924865d0 2024-05-08T22:37:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.620+0000 7f81537fe700 10 client.25166 handle_command_reply: tid=0 2024-05-08T22:37:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.620+0000 7f8186640b80 1 client.25166 shutdown 2024-05-08T22:37:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f8186640b80 1 -- 172.21.15.62:0/2910315891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55899247e9b0 msgr2=0x5589924933f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f8186640b80 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55899247e9b0 0x5589924933f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8164000c00 tx=0x7f8164009f50 comp rx=0 tx=0).stop 2024-05-08T22:37:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f8186640b80 1 -- 172.21.15.62:0/2910315891 shutdown_connections 2024-05-08T22:37:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f8186640b80 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x5589924865d0 0x5589924889f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f8186640b80 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558992480200 0x558992484a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f81537fe700 0 client.25166 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f8186640b80 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55899247e9b0 0x5589924933f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f8186640b80 1 --2- 172.21.15.62:0/2910315891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55899247f310 0x558992493930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f8186640b80 1 -- 172.21.15.62:0/2910315891 >> 172.21.15.62:0/2910315891 conn(0x5589923dd1b0 msgr2=0x558992490e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f8186640b80 1 -- 172.21.15.62:0/2910315891 shutdown_connections 2024-05-08T22:37:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f8186640b80 1 -- 172.21.15.62:0/2910315891 wait complete. 2024-05-08T22:37:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f8186640b80 20 client.25166 trim_cache size 0 max 16384 2024-05-08T22:37:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-05-08T22:37:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-05-08T22:37:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-05-08T22:37:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f817a486700 1 -- 172.21.15.62:0/748156525 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f815c0a1210 msgr2=0x7f815c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.621+0000 7f817a486700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f815c0a1210 0x7f815c0a36d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f816c009fd0 tx=0x7f816c0091c0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.622+0000 7f817a486700 1 -- 172.21.15.62:0/748156525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8174100f90 msgr2=0x7f81741973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.622+0000 7f817a486700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8174100f90 0x7f81741973e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f816800e9a0 tx=0x7f816800ed60 comp rx=0 tx=0).stop 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.622+0000 7f817a486700 1 -- 172.21.15.62:0/748156525 shutdown_connections 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.622+0000 7f817a486700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8174101930 0x7f817419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.622+0000 7f817a486700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f815c0a1210 0x7f815c0a36d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.622+0000 7f817a486700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81740ffda0 0x7f8174196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.622+0000 7f817a486700 1 --2- 172.21.15.62:0/748156525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8174100f90 0x7f81741973e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.622+0000 7f817a486700 1 -- 172.21.15.62:0/748156525 >> 172.21.15.62:0/748156525 conn(0x7f81740fb310 msgr2=0x7f8174104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.622+0000 7f817a486700 1 -- 172.21.15.62:0/748156525 shutdown_connections 2024-05-08T22:37:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:37.622+0000 7f817a486700 1 -- 172.21.15.62:0/748156525 wait complete. 2024-05-08T22:37:37.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:37 smithi066 ceph-mon[46830]: pgmap v297: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:37.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:37 smithi079 ceph-mon[46738]: pgmap v297: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:37.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:37 smithi062 ceph-mon[42571]: pgmap v297: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:39.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:39 smithi066 ceph-mon[46830]: pgmap v298: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:39.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:39 smithi079 ceph-mon[46738]: pgmap v298: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:39.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:39 smithi062 ceph-mon[42571]: pgmap v298: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:41.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:41 smithi066 ceph-mon[46830]: pgmap v299: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-08T22:37:41.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:41 smithi079 ceph-mon[46738]: pgmap v299: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-08T22:37:41.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:41 smithi062 ceph-mon[42571]: pgmap v299: 97 pgs: 97 active+clean; 700 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-08T22:37:43.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:43 smithi066 ceph-mon[46830]: pgmap v300: 97 pgs: 97 active+clean; 706 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:43.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:43 smithi079 ceph-mon[46738]: pgmap v300: 97 pgs: 97 active+clean; 706 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:43.915 INFO:tasks.workunit.client.0.smithi062.stderr:Updating files: 91% (11271/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-08T22:37:43.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:43 smithi062 ceph-mon[42571]: pgmap v300: 97 pgs: 97 active+clean; 706 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T22:37:44.871 INFO:tasks.workunit.client.0.smithi062.stderr:Note: switching to 'd21e4f961318b56f0d915caa2490c3711bf13c34'. 2024-05-08T22:37:44.871 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-05-08T22:37:44.871 INFO:tasks.workunit.client.0.smithi062.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T22:37:44.871 INFO:tasks.workunit.client.0.smithi062.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr: git switch -c 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr:Or undo this operation with: 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr: git switch - 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-05-08T22:37:44.872 INFO:tasks.workunit.client.0.smithi062.stderr:HEAD is now at d21e4f96131 Merge PR #57343 into wip-pdonnell-testing-20240508.150423-reef 2024-05-08T22:37:44.878 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T22:37:44.949 INFO:tasks.workunit.client.0.smithi062.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T22:37:44.951 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T22:37:44.951 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T22:37:45.437 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T22:37:45.532 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T22:37:45.571 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T22:37:45.572 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T22:37:45.572 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T22:37:45.611 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T22:37:45.614 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:37:45.614 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T22:37:45.672 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-05-08T22:37:45.673 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-05-08T22:37:45.673 DEBUG:teuthology.orchestra.run.smithi062:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-05-08T22:37:45.741 INFO:tasks.workunit.client.0.smithi062.stdout:test large (16MB) dio write 2024-05-08T22:37:45.741 INFO:tasks.workunit.client.0.smithi062.stderr:+ echo 'test large (16MB) dio write' 2024-05-08T22:37:45.741 INFO:tasks.workunit.client.0.smithi062.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-05-08T22:37:45.825 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:45 smithi066 ceph-mon[46830]: pgmap v301: 97 pgs: 97 active+clean; 706 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:45.857 INFO:tasks.workunit.client.0.smithi062.stderr:1+0 records in 2024-05-08T22:37:45.857 INFO:tasks.workunit.client.0.smithi062.stderr:1+0 records out 2024-05-08T22:37:45.857 INFO:tasks.workunit.client.0.smithi062.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.114009 s, 147 MB/s 2024-05-08T22:37:45.857 INFO:tasks.workunit.client.0.smithi062.stderr:+ echo OK 2024-05-08T22:37:45.858 INFO:tasks.workunit.client.0.smithi062.stdout:OK 2024-05-08T22:37:45.858 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T22:37:45.858 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T22:37:45.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:45 smithi079 ceph-mon[46738]: pgmap v301: 97 pgs: 97 active+clean; 706 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:45.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:45 smithi062 ceph-mon[42571]: pgmap v301: 97 pgs: 97 active+clean; 706 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:45.936 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-05-08T22:37:45.936 DEBUG:teuthology.orchestra.run.smithi062:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-05-08T22:37:47.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:47 smithi066 ceph-mon[46830]: pgmap v302: 97 pgs: 97 active+clean; 706 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:47.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:47 smithi079 ceph-mon[46738]: pgmap v302: 97 pgs: 97 active+clean; 706 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:47.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:47 smithi062 ceph-mon[42571]: pgmap v302: 97 pgs: 97 active+clean; 706 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T22:37:49.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:49 smithi066 ceph-mon[46830]: pgmap v303: 97 pgs: 97 active+clean; 8.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 684 KiB/s wr, 11 op/s 2024-05-08T22:37:49.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:49 smithi079 ceph-mon[46738]: pgmap v303: 97 pgs: 97 active+clean; 8.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 684 KiB/s wr, 11 op/s 2024-05-08T22:37:49.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:49 smithi062 ceph-mon[42571]: pgmap v303: 97 pgs: 97 active+clean; 8.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 684 KiB/s wr, 11 op/s 2024-05-08T22:37:50.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.775+0000 7fce07fc0700 1 -- 172.21.15.62:0/3141481709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce00100f90 msgr2=0x7fce001013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.775+0000 7fce07fc0700 1 --2- 172.21.15.62:0/3141481709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce00100f90 0x7fce001013f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00099c0 tx=0x7fcdf0009cd0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.775+0000 7fce07fc0700 1 -- 172.21.15.62:0/3141481709 shutdown_connections 2024-05-08T22:37:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.775+0000 7fce07fc0700 1 --2- 172.21.15.62:0/3141481709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce00101930 0x7fce001081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.775+0000 7fce07fc0700 1 --2- 172.21.15.62:0/3141481709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce000ffda0 0x7fce001001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.775+0000 7fce07fc0700 1 --2- 172.21.15.62:0/3141481709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce00100f90 0x7fce001013f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.775+0000 7fce07fc0700 1 -- 172.21.15.62:0/3141481709 >> 172.21.15.62:0/3141481709 conn(0x7fce000fb310 msgr2=0x7fce000fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.775+0000 7fce07fc0700 1 -- 172.21.15.62:0/3141481709 shutdown_connections 2024-05-08T22:37:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.775+0000 7fce07fc0700 1 -- 172.21.15.62:0/3141481709 wait complete. 2024-05-08T22:37:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce07fc0700 1 Processor -- start 2024-05-08T22:37:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce07fc0700 1 -- start start 2024-05-08T22:37:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce07fc0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000ffda0 0x7fce00196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce07fc0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce00100f90 0x7fce00197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce07fc0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce00101930 0x7fce0019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce07fc0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0019e8f0 con 0x7fce00101930 2024-05-08T22:37:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce07fc0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0019ea30 con 0x7fce00100f90 2024-05-08T22:37:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce07fc0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0019ebc0 con 0x7fce000ffda0 2024-05-08T22:37:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce0655d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce00101930 0x7fce0019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce0655d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce00101930 0x7fce0019e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42834/0 (socket says 172.21.15.62:42834) 2024-05-08T22:37:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce0655d700 1 -- 172.21.15.62:0/1260766330 learned_addr learned my addr 172.21.15.62:0/1260766330 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce0555b700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce00100f90 0x7fce00197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.776+0000 7fce05d5c700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000ffda0 0x7fce00196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.777+0000 7fce0655d700 1 -- 172.21.15.62:0/1260766330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000ffda0 msgr2=0x7fce00196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.777+0000 7fce0655d700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000ffda0 0x7fce00196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.777+0000 7fce0655d700 1 -- 172.21.15.62:0/1260766330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce00100f90 msgr2=0x7fce00197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.777+0000 7fce0655d700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce00100f90 0x7fce00197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.777+0000 7fce0655d700 1 -- 172.21.15.62:0/1260766330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf00095f0 con 0x7fce00101930 2024-05-08T22:37:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.777+0000 7fce0655d700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce00101930 0x7fce0019e320 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800ef30 tx=0x7fcdf800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.777+0000 7fcdf6ffd700 1 -- 172.21.15.62:0/1260766330 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8018070 con 0x7fce00101930 2024-05-08T22:37:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.777+0000 7fce07fc0700 1 -- 172.21.15.62:0/1260766330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0019ee40 con 0x7fce00101930 2024-05-08T22:37:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.777+0000 7fcdf6ffd700 1 -- 172.21.15.62:0/1260766330 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdf800f040 con 0x7fce00101930 2024-05-08T22:37:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.777+0000 7fcdf6ffd700 1 -- 172.21.15.62:0/1260766330 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf800cdb0 con 0x7fce00101930 2024-05-08T22:37:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.777+0000 7fce07fc0700 1 -- 172.21.15.62:0/1260766330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0019f2c0 con 0x7fce00101930 2024-05-08T22:37:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.780+0000 7fcdf6ffd700 1 -- 172.21.15.62:0/1260766330 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcdf8003a00 con 0x7fce00101930 2024-05-08T22:37:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.780+0000 7fcdf6ffd700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcdec0a1190 0x7fcdec0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fcdf6ffd700 1 -- 172.21.15.62:0/1260766330 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcdf80cc4a0 con 0x7fce00101930 2024-05-08T22:37:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fce1417ab80 1 Processor -- start 2024-05-08T22:37:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fce05d5c700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcdec0a1190 0x7fcdec0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fce1417ab80 1 -- start start 2024-05-08T22:37:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f54a80 0x562b34f54ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f553e0 0x562b34f55cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f56750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fce1417ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34eac0f0 con 0x562b34f562d0 2024-05-08T22:37:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fce1417ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34ea9ab0 con 0x562b34f553e0 2024-05-08T22:37:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fce1417ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34ea53a0 con 0x562b34f54a80 2024-05-08T22:37:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fce05d5c700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcdec0a1190 0x7fcdec0a3650 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fce00100df0 tx=0x7fcdfc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.781+0000 7fce0655d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f56750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce0655d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f56750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42844/0 (socket says 172.21.15.62:42844) 2024-05-08T22:37:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce0655d700 1 -- 172.21.15.62:0/2615900259 learned_addr learned my addr 172.21.15.62:0/2615900259 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce05d5c700 1 --2- 172.21.15.62:0/2615900259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f54a80 0x562b34f54ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce0555b700 1 --2- 172.21.15.62:0/2615900259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f553e0 0x562b34f55cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce0655d700 1 -- 172.21.15.62:0/2615900259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f54a80 msgr2=0x562b34f54ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce0655d700 1 --2- 172.21.15.62:0/2615900259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f54a80 0x562b34f54ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce0655d700 1 -- 172.21.15.62:0/2615900259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f553e0 msgr2=0x562b34f55cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce0655d700 1 --2- 172.21.15.62:0/2615900259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f553e0 0x562b34f55cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce0655d700 1 -- 172.21.15.62:0/2615900259 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b34d26350 con 0x562b34f562d0 2024-05-08T22:37:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce05d5c700 1 --2- 172.21.15.62:0/2615900259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f54a80 0x562b34f54ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce0555b700 1 --2- 172.21.15.62:0/2615900259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f553e0 0x562b34f55cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fce0655d700 1 --2- 172.21.15.62:0/2615900259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f56750 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8096400 tx=0x7fcdf8099830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb33ec2caee11fd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fcdebfff700 1 -- 172.21.15.62:0/2615900259 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf801b320 con 0x562b34f562d0 2024-05-08T22:37:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fcdebfff700 1 -- 172.21.15.62:0/2615900259 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdf809a090 con 0x562b34f562d0 2024-05-08T22:37:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.782+0000 7fcdebfff700 1 -- 172.21.15.62:0/2615900259 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8097d30 con 0x562b34f562d0 2024-05-08T22:37:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.783+0000 7fce1417ab80 1 -- 172.21.15.62:0/2615900259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 msgr2=0x562b34f56750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.783+0000 7fce1417ab80 1 --2- 172.21.15.62:0/2615900259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f56750 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8096400 tx=0x7fcdf8099830 comp rx=0 tx=0).stop 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.783+0000 7fce1417ab80 1 -- 172.21.15.62:0/2615900259 shutdown_connections 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.783+0000 7fce1417ab80 1 --2- 172.21.15.62:0/2615900259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f56750 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.783+0000 7fce1417ab80 1 --2- 172.21.15.62:0/2615900259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f54a80 0x562b34f54ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.783+0000 7fce1417ab80 1 --2- 172.21.15.62:0/2615900259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f553e0 0x562b34f55cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.783+0000 7fce1417ab80 1 -- 172.21.15.62:0/2615900259 >> 172.21.15.62:0/2615900259 conn(0x562b34eb31b0 msgr2=0x562b34f51440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.783+0000 7fce1417ab80 1 -- 172.21.15.62:0/2615900259 shutdown_connections 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.783+0000 7fce1417ab80 1 -- 172.21.15.62:0/2615900259 wait complete. 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce1417ab80 1 -- start start 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce1417ab80 1 Processor -- start 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 0x562b34f5e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f553e0 0x562b34f5ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f65e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce1417ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34e41500 con 0x562b34f562d0 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce1417ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34e411a0 con 0x562b34f54a80 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce1417ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34d28640 con 0x562b34f553e0 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce1417ab80 10 client.0 _ll_register_callbacks cb 0x562b34e4aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce0655d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f65e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce0655d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f65e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42856/0 (socket says 172.21.15.62:42856) 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce0655d700 1 -- 172.21.15.62:0/1015262294 learned_addr learned my addr 172.21.15.62:0/1015262294 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.784+0000 7fce05d5c700 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 0x562b34f5e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fce0655d700 1 -- 172.21.15.62:0/1015262294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f553e0 msgr2=0x562b34f5ee40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fce0655d700 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f553e0 0x562b34f5ee40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fce0655d700 1 -- 172.21.15.62:0/1015262294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 msgr2=0x562b34f5e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fce0655d700 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 0x562b34f5e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fce0655d700 1 -- 172.21.15.62:0/1015262294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce00108b30 con 0x562b34f562d0 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fce05d5c700 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 0x562b34f5e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fce0655d700 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f65e80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80ca000 tx=0x7fcdf8004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8701cb4af36e47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fcde97fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fcde97fa700 1 -- 172.21.15.62:0/1015262294 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80b6070 con 0x562b34f562d0 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fcde97fa700 1 -- 172.21.15.62:0/1015262294 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcdf809d7c0 con 0x562b34f562d0 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fce1417ab80 1 -- 172.21.15.62:0/1015262294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b34dfe210 con 0x562b34f562d0 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fcde97fa700 1 -- 172.21.15.62:0/1015262294 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdf80b22a0 con 0x562b34f562d0 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fce1417ab80 10 client.15480 fetch_fsmap learned FSMap version 29 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fce1417ab80 1 -- 172.21.15.62:0/1015262294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fce0019f2c0 con 0x562b34f562d0 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.785+0000 7fcde97fa700 1 -- 172.21.15.62:0/1015262294 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562b34dfe210 con 0x562b34f562d0 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.786+0000 7fcde97fa700 1 -- 172.21.15.62:0/1015262294 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcdf8081020 con 0x562b34f562d0 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.786+0000 7fce1417ab80 10 client.15480 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:37:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.786+0000 7fce1417ab80 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:37:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.786+0000 7fce1417ab80 20 client.15480 populate_metadata read hostname 'smithi062' 2024-05-08T22:37:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.786+0000 7fce1417ab80 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x562b34f6a370 0x562b34f6c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.786+0000 7fce1417ab80 4 client.15480 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:37:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.786+0000 7fce1417ab80 1 -- 172.21.15.62:0/1015262294 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b34f247b0 con 0x562b34f6a370 2024-05-08T22:37:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.786+0000 7fce05d5c700 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x562b34f6a370 0x562b34f6c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fcde97fa700 10 client.15480 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fce05d5c700 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x562b34f6a370 0x562b34f6c790 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2304eb8123efbd82 server_cookie=5094c3eb6b44c62 in_seq=0 out_seq=0 2024-05-08T22:37:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fcde97fa700 1 -- 172.21.15.62:0/1015262294 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b34f247b0 con 0x562b34f6a370 2024-05-08T22:37:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fcde97fa700 10 client.15480 handle_command_reply: tid=0 2024-05-08T22:37:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fce1417ab80 1 client.15480 shutdown 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fce1417ab80 1 -- 172.21.15.62:0/1015262294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 msgr2=0x562b34f65e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fce1417ab80 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f65e80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80ca000 tx=0x7fcdf8004b90 comp rx=0 tx=0).stop 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fce1417ab80 1 -- 172.21.15.62:0/1015262294 shutdown_connections 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fce1417ab80 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x562b34f6a370 0x562b34f6c790 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fce1417ab80 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f562d0 0x562b34f65e80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fcde97fa700 0 client.15480 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fce1417ab80 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 0x562b34f5e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fce1417ab80 1 --2- 172.21.15.62:0/1015262294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f553e0 0x562b34f5ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.787+0000 7fce1417ab80 1 -- 172.21.15.62:0/1015262294 >> 172.21.15.62:0/1015262294 conn(0x562b34eb31b0 msgr2=0x562b34f4f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.788+0000 7fce1417ab80 1 -- 172.21.15.62:0/1015262294 shutdown_connections 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.788+0000 7fce1417ab80 1 -- 172.21.15.62:0/1015262294 wait complete. 2024-05-08T22:37:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.788+0000 7fce1417ab80 20 client.15480 trim_cache size 0 max 16384 2024-05-08T22:37:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.803+0000 7fce1417ab80 1 Processor -- start 2024-05-08T22:37:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.803+0000 7fce1417ab80 1 -- start start 2024-05-08T22:37:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f54a80 0x562b34f54e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f553e0 0x562b34f65e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f562d0 0x562b34f566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce1417ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34f5f930 con 0x562b34f54a80 2024-05-08T22:37:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce1417ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34f51730 con 0x562b34f553e0 2024-05-08T22:37:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce1417ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34f5faf0 con 0x562b34f562d0 2024-05-08T22:37:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce05d5c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f54a80 0x562b34f54e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce05d5c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f54a80 0x562b34f54e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42858/0 (socket says 172.21.15.62:42858) 2024-05-08T22:37:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce05d5c700 1 -- 172.21.15.62:0/2607657894 learned_addr learned my addr 172.21.15.62:0/2607657894 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce0655d700 1 --2- 172.21.15.62:0/2607657894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f562d0 0x562b34f566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce0555b700 1 --2- 172.21.15.62:0/2607657894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f553e0 0x562b34f65e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce05d5c700 1 -- 172.21.15.62:0/2607657894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f562d0 msgr2=0x562b34f566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce05d5c700 1 --2- 172.21.15.62:0/2607657894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f562d0 0x562b34f566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce05d5c700 1 -- 172.21.15.62:0/2607657894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f553e0 msgr2=0x562b34f65e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce05d5c700 1 --2- 172.21.15.62:0/2607657894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f553e0 0x562b34f65e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce05d5c700 1 -- 172.21.15.62:0/2607657894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b34cd2b00 con 0x562b34f54a80 2024-05-08T22:37:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fce05d5c700 1 --2- 172.21.15.62:0/2607657894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f54a80 0x562b34f54e60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0097d0 tx=0x7fcdfc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3934b9da87dc19 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.804+0000 7fcdf4ff9700 1 -- 172.21.15.62:0/2607657894 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc025040 con 0x562b34f54a80 2024-05-08T22:37:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.805+0000 7fcdf4ff9700 1 -- 172.21.15.62:0/2607657894 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdfc016070 con 0x562b34f54a80 2024-05-08T22:37:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.805+0000 7fcdf4ff9700 1 -- 172.21.15.62:0/2607657894 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0038c0 con 0x562b34f54a80 2024-05-08T22:37:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.805+0000 7fce1417ab80 1 -- 172.21.15.62:0/2607657894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f54a80 msgr2=0x562b34f54e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.805+0000 7fce1417ab80 1 --2- 172.21.15.62:0/2607657894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f54a80 0x562b34f54e60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0097d0 tx=0x7fcdfc014000 comp rx=0 tx=0).stop 2024-05-08T22:37:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.805+0000 7fce1417ab80 1 -- 172.21.15.62:0/2607657894 shutdown_connections 2024-05-08T22:37:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.805+0000 7fce1417ab80 1 --2- 172.21.15.62:0/2607657894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f562d0 0x562b34f566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.805+0000 7fce1417ab80 1 --2- 172.21.15.62:0/2607657894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f54a80 0x562b34f54e60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.805+0000 7fce1417ab80 1 --2- 172.21.15.62:0/2607657894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f553e0 0x562b34f65e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.805+0000 7fce1417ab80 1 -- 172.21.15.62:0/2607657894 >> 172.21.15.62:0/2607657894 conn(0x562b34eb31b0 msgr2=0x562b34f51020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.805+0000 7fce1417ab80 1 -- 172.21.15.62:0/2607657894 shutdown_connections 2024-05-08T22:37:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.806+0000 7fce1417ab80 1 -- 172.21.15.62:0/2607657894 wait complete. 2024-05-08T22:37:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.806+0000 7fce1417ab80 1 -- start start 2024-05-08T22:37:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.806+0000 7fce1417ab80 1 Processor -- start 2024-05-08T22:37:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.806+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 0x562b34f63900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.806+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f553e0 0x562b34f63e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.806+0000 7fce1417ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f562d0 0x562b34f64db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.806+0000 7fce1417ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34f652f0 con 0x562b34f553e0 2024-05-08T22:37:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.806+0000 7fce1417ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34f65460 con 0x562b34f54a80 2024-05-08T22:37:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.806+0000 7fce1417ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34f65620 con 0x562b34f562d0 2024-05-08T22:37:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.806+0000 7fce1417ab80 10 client.0 _ll_register_callbacks cb 0x562b34e4aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:37:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce0555b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f553e0 0x562b34f63e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce05d5c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 0x562b34f63900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce0555b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f553e0 0x562b34f63e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42862/0 (socket says 172.21.15.62:42862) 2024-05-08T22:37:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce0555b700 1 -- 172.21.15.62:0/3466550965 learned_addr learned my addr 172.21.15.62:0/3466550965 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:37:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce0655d700 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f562d0 0x562b34f64db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce0555b700 1 -- 172.21.15.62:0/3466550965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f562d0 msgr2=0x562b34f64db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce0555b700 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f562d0 0x562b34f64db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce0555b700 1 -- 172.21.15.62:0/3466550965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 msgr2=0x562b34f63900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:37:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce0555b700 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 0x562b34f63900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce0555b700 1 -- 172.21.15.62:0/3466550965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc011a60 con 0x562b34f553e0 2024-05-08T22:37:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce05d5c700 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 0x562b34f63900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:37:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fce0555b700 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f553e0 0x562b34f63e40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000b580 tx=0x7fcdf0005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35900a0230f7787 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:37:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fcde97fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:37:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fcde97fa700 1 -- 172.21.15.62:0/3466550965 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf000bc20 con 0x562b34f553e0 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.807+0000 7fcde97fa700 1 -- 172.21.15.62:0/3466550965 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcdf000bd80 con 0x562b34f553e0 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fce1417ab80 1 -- 172.21.15.62:0/3466550965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdf809a090 con 0x562b34f553e0 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fcde97fa700 1 -- 172.21.15.62:0/3466550965 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdf00037d0 con 0x562b34f553e0 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fce1417ab80 10 client.15486 fetch_fsmap learned FSMap version 29 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fce1417ab80 1 -- 172.21.15.62:0/3466550965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562b34e6b500 con 0x562b34f553e0 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fcde97fa700 1 -- 172.21.15.62:0/3466550965 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdf809a090 con 0x562b34f553e0 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fcde97fa700 1 -- 172.21.15.62:0/3466550965 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcdf0024d00 con 0x562b34f553e0 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fce1417ab80 10 client.15486 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fce1417ab80 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fce1417ab80 20 client.15486 populate_metadata read hostname 'smithi062' 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fce1417ab80 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x562b34fc1500 0x562b34fc3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:37:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fce1417ab80 4 client.15486 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:37:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.808+0000 7fce1417ab80 1 -- 172.21.15.62:0/3466550965 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b34e9de60 con 0x562b34fc1500 2024-05-08T22:37:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.809+0000 7fce05d5c700 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x562b34fc1500 0x562b34fc3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:37:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.809+0000 7fcde97fa700 10 client.15486 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.809+0000 7fce05d5c700 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x562b34fc1500 0x562b34fc3920 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f49cf74d87eff1f3 server_cookie=28f328ad441af64f in_seq=0 out_seq=0 2024-05-08T22:37:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fcde97fa700 1 -- 172.21.15.62:0/3466550965 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b34e9de60 con 0x562b34fc1500 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fcde97fa700 10 client.15486 handle_command_reply: tid=0 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fce1417ab80 1 client.15486 shutdown 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fce1417ab80 1 -- 172.21.15.62:0/3466550965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f553e0 msgr2=0x562b34f63e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fce1417ab80 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f553e0 0x562b34f63e40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000b580 tx=0x7fcdf0005790 comp rx=0 tx=0).stop 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fce1417ab80 1 -- 172.21.15.62:0/3466550965 shutdown_connections 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fce1417ab80 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x562b34fc1500 0x562b34fc3920 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fce1417ab80 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b34f562d0 0x562b34f64db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fce1417ab80 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562b34f54a80 0x562b34f63900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fce1417ab80 1 --2- 172.21.15.62:0/3466550965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562b34f553e0 0x562b34f63e40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fcde97fa700 0 client.15486 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fce1417ab80 1 -- 172.21.15.62:0/3466550965 >> 172.21.15.62:0/3466550965 conn(0x562b34eb31b0 msgr2=0x562b34f5dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fce1417ab80 1 -- 172.21.15.62:0/3466550965 shutdown_connections 2024-05-08T22:37:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.815+0000 7fce1417ab80 1 -- 172.21.15.62:0/3466550965 wait complete. 2024-05-08T22:37:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce1417ab80 20 client.15486 trim_cache size 0 max 16384 2024-05-08T22:37:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-05-08T22:37:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-05-08T22:37:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-05-08T22:37:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-05-08T22:37:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 -- 172.21.15.62:0/1260766330 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcdec0a1190 msgr2=0x7fcdec0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcdec0a1190 0x7fcdec0a3650 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fce00100df0 tx=0x7fcdfc009260 comp rx=0 tx=0).stop 2024-05-08T22:37:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 -- 172.21.15.62:0/1260766330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce00101930 msgr2=0x7fce0019e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:37:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce00101930 0x7fce0019e320 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800ef30 tx=0x7fcdf800c3a0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 -- 172.21.15.62:0/1260766330 shutdown_connections 2024-05-08T22:37:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce00101930 0x7fce0019e320 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000ffda0 0x7fce00196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fcdec0a1190 0x7fcdec0a3650 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 --2- 172.21.15.62:0/1260766330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce00100f90 0x7fce00197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:37:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 -- 172.21.15.62:0/1260766330 >> 172.21.15.62:0/1260766330 conn(0x7fce000fb310 msgr2=0x7fce000fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:37:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 -- 172.21.15.62:0/1260766330 shutdown_connections 2024-05-08T22:37:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:37:50.816+0000 7fce07fc0700 1 -- 172.21.15.62:0/1260766330 wait complete. 2024-05-08T22:37:51.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:51 smithi066 ceph-mon[46830]: pgmap v304: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 1.07k op/s 2024-05-08T22:37:51.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:51 smithi079 ceph-mon[46738]: pgmap v304: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 1.07k op/s 2024-05-08T22:37:51.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:51 smithi062 ceph-mon[42571]: pgmap v304: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 1.07k op/s 2024-05-08T22:37:53.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:53 smithi066 ceph-mon[46830]: pgmap v305: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 1.07k op/s 2024-05-08T22:37:53.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:53 smithi079 ceph-mon[46738]: pgmap v305: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 1.07k op/s 2024-05-08T22:37:53.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:53 smithi062 ceph-mon[42571]: pgmap v305: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 1.07k op/s 2024-05-08T22:37:55.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:55 smithi066 ceph-mon[46830]: pgmap v306: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.07k op/s 2024-05-08T22:37:55.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:55 smithi079 ceph-mon[46738]: pgmap v306: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.07k op/s 2024-05-08T22:37:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:55 smithi062 ceph-mon[42571]: pgmap v306: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.07k op/s 2024-05-08T22:37:55.999 INFO:tasks.workunit.client.0.smithi062.stdout:.......... 2024-05-08T22:37:55.999 INFO:tasks.workunit.client.0.smithi062.stdout:do_o_direct_reads: SUCCESS 2024-05-08T22:37:56.003 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T22:37:56.003 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T22:37:56.040 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-05-08T22:37:56.040 DEBUG:teuthology.orchestra.run.smithi062:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-05-08T22:37:56.104 INFO:tasks.workunit.client.0.smithi062.stdout:test read from hole 2024-05-08T22:37:56.105 INFO:tasks.workunit.client.0.smithi062.stderr:+ echo 'test read from hole' 2024-05-08T22:37:56.105 INFO:tasks.workunit.client.0.smithi062.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-05-08T22:37:56.106 INFO:tasks.workunit.client.0.smithi062.stderr:0+0 records in 2024-05-08T22:37:56.106 INFO:tasks.workunit.client.0.smithi062.stderr:0+0 records out 2024-05-08T22:37:56.107 INFO:tasks.workunit.client.0.smithi062.stderr:0 bytes copied, 3.4672e-05 s, 0.0 kB/s 2024-05-08T22:37:56.107 INFO:tasks.workunit.client.0.smithi062.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-05-08T22:37:56.108 INFO:tasks.workunit.client.0.smithi062.stderr:2+0 records in 2024-05-08T22:37:56.108 INFO:tasks.workunit.client.0.smithi062.stderr:2+0 records out 2024-05-08T22:37:56.108 INFO:tasks.workunit.client.0.smithi062.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000665565 s, 1.5 MB/s 2024-05-08T22:37:56.108 INFO:tasks.workunit.client.0.smithi062.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-05-08T22:37:56.108 INFO:tasks.workunit.client.0.smithi062.stderr:2+0 records in 2024-05-08T22:37:56.109 INFO:tasks.workunit.client.0.smithi062.stderr:2+0 records out 2024-05-08T22:37:56.109 INFO:tasks.workunit.client.0.smithi062.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 5.3531e-05 s, 19.1 MB/s 2024-05-08T22:37:56.109 INFO:tasks.workunit.client.0.smithi062.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-05-08T22:37:56.109 INFO:tasks.workunit.client.0.smithi062.stdout:other thing 2024-05-08T22:37:56.110 INFO:tasks.workunit.client.0.smithi062.stderr:+ echo 'other thing' 2024-05-08T22:37:56.110 INFO:tasks.workunit.client.0.smithi062.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-05-08T22:37:56.110 INFO:tasks.workunit.client.0.smithi062.stderr:1+0 records in 2024-05-08T22:37:56.110 INFO:tasks.workunit.client.0.smithi062.stderr:1+0 records out 2024-05-08T22:37:56.110 INFO:tasks.workunit.client.0.smithi062.stderr:500 bytes copied, 3.9551e-05 s, 12.6 MB/s 2024-05-08T22:37:56.110 INFO:tasks.workunit.client.0.smithi062.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-05-08T22:37:56.116 INFO:tasks.workunit.client.0.smithi062.stderr:0+1 records in 2024-05-08T22:37:56.116 INFO:tasks.workunit.client.0.smithi062.stderr:0+1 records out 2024-05-08T22:37:56.117 INFO:tasks.workunit.client.0.smithi062.stderr:500 bytes copied, 0.00451526 s, 111 kB/s 2024-05-08T22:37:56.117 INFO:tasks.workunit.client.0.smithi062.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-05-08T22:37:56.119 INFO:tasks.workunit.client.0.smithi062.stderr:0+1 records in 2024-05-08T22:37:56.119 INFO:tasks.workunit.client.0.smithi062.stderr:0+1 records out 2024-05-08T22:37:56.120 INFO:tasks.workunit.client.0.smithi062.stderr:500 bytes copied, 0.00225972 s, 221 kB/s 2024-05-08T22:37:56.120 INFO:tasks.workunit.client.0.smithi062.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-05-08T22:37:56.120 INFO:tasks.workunit.client.0.smithi062.stdout:OK 2024-05-08T22:37:56.121 INFO:tasks.workunit.client.0.smithi062.stderr:+ echo OK 2024-05-08T22:37:56.121 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T22:37:56.121 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T22:37:56.202 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-05-08T22:37:56.202 DEBUG:teuthology.orchestra.run.smithi062:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-05-08T22:37:56.276 INFO:tasks.workunit.client.0.smithi062.stdout:writing first 3 bytes of 10k file 2024-05-08T22:37:56.276 INFO:tasks.workunit.client.0.smithi062.stdout:reading O_DIRECT 2024-05-08T22:37:56.276 INFO:tasks.workunit.client.0.smithi062.stdout:got 10000 2024-05-08T22:37:56.277 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T22:37:56.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T22:37:56.354 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-05-08T22:37:56.354 DEBUG:teuthology.orchestra.run.smithi062:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-05-08T22:37:57.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:57 smithi066 ceph-mon[46830]: pgmap v307: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.07k op/s 2024-05-08T22:37:57.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:57 smithi079 ceph-mon[46738]: pgmap v307: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.07k op/s 2024-05-08T22:37:57.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:57 smithi062 ceph-mon[42571]: pgmap v307: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.07k op/s 2024-05-08T22:37:57.949 INFO:tasks.workunit.client.0.smithi062.stdout:writing pattern 2024-05-08T22:37:57.949 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-05-08T22:37:57.949 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-05-08T22:37:57.949 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-05-08T22:37:57.949 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-05-08T22:37:57.949 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-05-08T22:37:57.949 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-05-08T22:37:57.949 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-05-08T22:37:57.950 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-05-08T22:37:57.951 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-05-08T22:37:57.952 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-05-08T22:37:57.952 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-05-08T22:37:57.952 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-05-08T22:37:57.952 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-05-08T22:37:57.952 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-05-08T22:37:57.952 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-05-08T22:37:57.952 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-05-08T22:37:57.952 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-05-08T22:37:57.952 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-05-08T22:37:57.952 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-05-08T22:37:57.953 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-05-08T22:37:57.954 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-05-08T22:37:57.955 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-05-08T22:37:57.975 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-05-08T22:37:57.975 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-05-08T22:37:57.975 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-05-08T22:37:57.975 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-05-08T22:37:57.975 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-05-08T22:37:57.975 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-05-08T22:37:57.975 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-05-08T22:37:57.975 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-05-08T22:37:57.975 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-05-08T22:37:57.976 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-05-08T22:37:57.977 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-05-08T22:37:57.978 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-05-08T22:37:57.979 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-05-08T22:37:57.980 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-05-08T22:37:57.981 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-05-08T22:37:57.981 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-05-08T22:37:57.981 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-05-08T22:37:57.981 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-05-08T22:37:57.981 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-05-08T22:37:57.981 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-05-08T22:37:57.981 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-05-08T22:37:57.981 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-05-08T22:37:57.986 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-05-08T22:37:57.987 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-05-08T22:37:57.988 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-05-08T22:37:57.989 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-05-08T22:37:57.990 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-05-08T22:37:57.991 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-05-08T22:37:57.992 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-05-08T22:37:57.999 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-05-08T22:37:58.000 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-05-08T22:37:58.001 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-05-08T22:37:58.002 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-05-08T22:37:58.003 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-05-08T22:37:58.004 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-05-08T22:37:58.005 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-05-08T22:37:58.005 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-05-08T22:37:58.005 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-05-08T22:37:58.005 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-05-08T22:37:58.005 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-05-08T22:37:58.005 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-05-08T22:37:58.005 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-05-08T22:37:58.019 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-05-08T22:37:58.019 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-05-08T22:37:58.019 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-05-08T22:37:58.019 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-05-08T22:37:58.019 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-05-08T22:37:58.020 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-05-08T22:37:58.021 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-05-08T22:37:58.022 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-05-08T22:37:58.023 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-05-08T22:37:58.024 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-05-08T22:37:58.025 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-05-08T22:37:58.025 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-05-08T22:37:58.025 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-05-08T22:37:58.025 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-05-08T22:37:58.025 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-05-08T22:37:58.025 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-05-08T22:37:58.025 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-05-08T22:37:58.025 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-05-08T22:37:58.025 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-05-08T22:37:58.035 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-05-08T22:37:58.035 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-05-08T22:37:58.036 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-05-08T22:37:58.037 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-05-08T22:37:58.038 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-05-08T22:37:58.039 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-05-08T22:37:58.040 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-05-08T22:37:58.041 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-05-08T22:37:58.041 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-05-08T22:37:58.041 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-05-08T22:37:58.041 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-05-08T22:37:58.041 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-05-08T22:37:58.041 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-05-08T22:37:58.041 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-05-08T22:37:58.041 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-05-08T22:37:58.041 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-05-08T22:37:58.041 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-05-08T22:37:58.041 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-05-08T22:37:58.045 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-05-08T22:37:58.045 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-05-08T22:37:58.046 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-05-08T22:37:58.047 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-05-08T22:37:58.048 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-05-08T22:37:58.049 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-05-08T22:37:58.050 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-05-08T22:37:58.051 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-05-08T22:37:58.051 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-05-08T22:37:58.051 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-05-08T22:37:58.051 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-05-08T22:37:58.051 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-05-08T22:37:58.051 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-05-08T22:37:58.051 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-05-08T22:37:58.051 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-05-08T22:37:58.051 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-05-08T22:37:58.058 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-05-08T22:37:58.058 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-05-08T22:37:58.059 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-05-08T22:37:58.060 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-05-08T22:37:58.061 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-05-08T22:37:58.062 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-05-08T22:37:58.063 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-05-08T22:37:58.064 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-05-08T22:37:58.064 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-05-08T22:37:58.064 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-05-08T22:37:58.064 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-05-08T22:37:58.064 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-05-08T22:37:58.064 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-05-08T22:37:58.064 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-05-08T22:37:58.064 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-05-08T22:37:58.064 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-05-08T22:37:58.077 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-05-08T22:37:58.077 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-05-08T22:37:58.077 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-05-08T22:37:58.077 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-05-08T22:37:58.077 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-05-08T22:37:58.077 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-05-08T22:37:58.077 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-05-08T22:37:58.077 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-05-08T22:37:58.077 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-05-08T22:37:58.077 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-05-08T22:37:58.077 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-05-08T22:37:58.078 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-05-08T22:37:58.079 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-05-08T22:37:58.080 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-05-08T22:37:58.081 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-05-08T22:37:58.082 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-05-08T22:37:58.092 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-05-08T22:37:58.092 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-05-08T22:37:58.092 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-05-08T22:37:58.092 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-05-08T22:37:58.092 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-05-08T22:37:58.092 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-05-08T22:37:58.092 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-05-08T22:37:58.092 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-05-08T22:37:58.092 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-05-08T22:37:58.092 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-05-08T22:37:58.093 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-05-08T22:37:58.094 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-05-08T22:37:58.095 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-05-08T22:37:58.096 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-05-08T22:37:58.097 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-05-08T22:37:58.102 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-05-08T22:37:58.102 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-05-08T22:37:58.103 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-05-08T22:37:58.103 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-05-08T22:37:58.103 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-05-08T22:37:58.103 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-05-08T22:37:58.103 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-05-08T22:37:58.103 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-05-08T22:37:58.103 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-05-08T22:37:58.103 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-05-08T22:37:58.105 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-05-08T22:37:58.105 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-05-08T22:37:58.105 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-05-08T22:37:58.105 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-05-08T22:37:58.105 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-05-08T22:37:58.106 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-05-08T22:37:58.107 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-05-08T22:37:58.108 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-05-08T22:37:58.109 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-05-08T22:37:58.110 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-05-08T22:37:58.110 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-05-08T22:37:58.110 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-05-08T22:37:58.110 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-05-08T22:37:58.110 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-05-08T22:37:58.110 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-05-08T22:37:58.110 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-05-08T22:37:58.110 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-05-08T22:37:58.116 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-05-08T22:37:58.116 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-05-08T22:37:58.117 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-05-08T22:37:58.118 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-05-08T22:37:58.118 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-05-08T22:37:58.118 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-05-08T22:37:58.118 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-05-08T22:37:58.118 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-05-08T22:37:58.118 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-05-08T22:37:58.118 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-05-08T22:37:58.118 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-05-08T22:37:58.118 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-05-08T22:37:58.118 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-05-08T22:37:58.118 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-05-08T22:37:58.119 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-05-08T22:37:58.120 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-05-08T22:37:58.121 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-05-08T22:37:58.122 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-05-08T22:37:58.137 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-05-08T22:37:58.137 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-05-08T22:37:58.138 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-05-08T22:37:58.139 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-05-08T22:37:58.140 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-05-08T22:37:58.140 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-05-08T22:37:58.140 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-05-08T22:37:58.140 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-05-08T22:37:58.140 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-05-08T22:37:58.140 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-05-08T22:37:58.140 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-05-08T22:37:58.140 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-05-08T22:37:58.140 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-05-08T22:37:58.141 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-05-08T22:37:58.142 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-05-08T22:37:58.143 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-05-08T22:37:58.144 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-05-08T22:37:58.144 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-05-08T22:37:58.153 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-05-08T22:37:58.153 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-05-08T22:37:58.153 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-05-08T22:37:58.153 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-05-08T22:37:58.153 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-05-08T22:37:58.154 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-05-08T22:37:58.155 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-05-08T22:37:58.156 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-05-08T22:37:58.157 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-05-08T22:37:58.158 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-05-08T22:37:58.158 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-05-08T22:37:58.158 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-05-08T22:37:58.158 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-05-08T22:37:58.159 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-05-08T22:37:58.160 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-05-08T22:37:58.160 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-05-08T22:37:58.160 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-05-08T22:37:58.160 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-05-08T22:37:58.160 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-05-08T22:37:58.160 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-05-08T22:37:58.160 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-05-08T22:37:58.160 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-05-08T22:37:58.160 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-05-08T22:37:58.163 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-05-08T22:37:58.163 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-05-08T22:37:58.163 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-05-08T22:37:58.163 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-05-08T22:37:58.163 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-05-08T22:37:58.163 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-05-08T22:37:58.163 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-05-08T22:37:58.163 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-05-08T22:37:58.163 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-05-08T22:37:58.163 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-05-08T22:37:58.163 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-05-08T22:37:58.164 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-05-08T22:37:58.164 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-05-08T22:37:58.164 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-05-08T22:37:58.164 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-05-08T22:37:58.164 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-05-08T22:37:58.164 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-05-08T22:37:58.164 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-05-08T22:37:58.164 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-05-08T22:37:58.164 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-05-08T22:37:58.164 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-05-08T22:37:58.164 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-05-08T22:37:58.165 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-05-08T22:37:58.166 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-05-08T22:37:58.167 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-05-08T22:37:58.167 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-05-08T22:37:58.167 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-05-08T22:37:58.167 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-05-08T22:37:58.167 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-05-08T22:37:58.167 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-05-08T22:37:58.167 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-05-08T22:37:58.167 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-05-08T22:37:58.167 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-05-08T22:37:58.167 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-05-08T22:37:58.167 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-05-08T22:37:58.168 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-05-08T22:37:58.168 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-05-08T22:37:58.168 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-05-08T22:37:58.168 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-05-08T22:37:58.169 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-05-08T22:37:58.169 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-05-08T22:37:58.169 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-05-08T22:37:58.169 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-05-08T22:37:58.170 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-05-08T22:37:58.170 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-05-08T22:37:58.170 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-05-08T22:37:58.170 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-05-08T22:37:58.170 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-05-08T22:37:58.170 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-05-08T22:37:58.170 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-05-08T22:37:58.171 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-05-08T22:37:58.171 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-05-08T22:37:58.171 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-05-08T22:37:58.171 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-05-08T22:37:58.171 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-05-08T22:37:58.171 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-05-08T22:37:58.177 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-05-08T22:37:58.178 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-05-08T22:37:58.178 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-05-08T22:37:58.178 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-05-08T22:37:58.178 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-05-08T22:37:58.178 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-05-08T22:37:58.178 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-05-08T22:37:58.178 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-05-08T22:37:58.179 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-05-08T22:37:58.179 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-05-08T22:37:58.180 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-05-08T22:37:58.181 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-05-08T22:37:58.182 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-05-08T22:37:58.182 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-05-08T22:37:58.182 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-05-08T22:37:58.182 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-05-08T22:37:58.182 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-05-08T22:37:58.182 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-05-08T22:37:58.182 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-05-08T22:37:58.182 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-05-08T22:37:58.182 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-05-08T22:37:58.183 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-05-08T22:37:58.183 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-05-08T22:37:58.183 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-05-08T22:37:58.183 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-05-08T22:37:58.183 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-05-08T22:37:58.183 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-05-08T22:37:58.184 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-05-08T22:37:58.184 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-05-08T22:37:58.184 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-05-08T22:37:58.184 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-05-08T22:37:58.185 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-05-08T22:37:58.185 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-05-08T22:37:58.185 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-05-08T22:37:58.185 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-05-08T22:37:58.185 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-05-08T22:37:58.185 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-05-08T22:37:58.186 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-05-08T22:37:58.186 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-05-08T22:37:58.186 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-05-08T22:37:58.186 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-05-08T22:37:58.186 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-05-08T22:37:58.186 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-05-08T22:37:58.187 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-05-08T22:37:58.187 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-05-08T22:37:58.187 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-05-08T22:37:58.187 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-05-08T22:37:58.187 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-05-08T22:37:58.188 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-05-08T22:37:58.188 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-05-08T22:37:58.188 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-05-08T22:37:58.188 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-05-08T22:37:58.188 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-05-08T22:37:58.188 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-05-08T22:37:58.189 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-05-08T22:37:58.190 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-05-08T22:37:58.190 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-05-08T22:37:58.190 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-05-08T22:37:58.190 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-05-08T22:37:58.190 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-05-08T22:37:58.190 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-05-08T22:37:58.196 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-05-08T22:37:58.196 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-05-08T22:37:58.196 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-05-08T22:37:58.196 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-05-08T22:37:58.197 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-05-08T22:37:58.198 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-05-08T22:37:58.199 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-05-08T22:37:58.200 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-05-08T22:37:58.201 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-05-08T22:37:58.202 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-05-08T22:37:58.202 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-05-08T22:37:58.202 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-05-08T22:37:58.202 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-05-08T22:37:58.202 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-05-08T22:37:58.208 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-05-08T22:37:58.208 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-05-08T22:37:58.208 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-05-08T22:37:58.209 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-05-08T22:37:58.210 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-05-08T22:37:58.211 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-05-08T22:37:58.212 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-05-08T22:37:58.212 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-05-08T22:37:58.212 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-05-08T22:37:58.212 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-05-08T22:37:58.212 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-05-08T22:37:58.212 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-05-08T22:37:58.212 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-05-08T22:37:58.212 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-05-08T22:37:58.212 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-05-08T22:37:58.212 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-05-08T22:37:58.212 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-05-08T22:37:58.213 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-05-08T22:37:58.214 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-05-08T22:37:58.214 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-05-08T22:37:58.214 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-05-08T22:37:58.214 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-05-08T22:37:58.214 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-05-08T22:37:58.214 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-05-08T22:37:58.214 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-05-08T22:37:58.214 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-05-08T22:37:58.214 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-05-08T22:37:58.218 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-05-08T22:37:58.218 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-05-08T22:37:58.218 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-05-08T22:37:58.219 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-05-08T22:37:58.220 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-05-08T22:37:58.221 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-05-08T22:37:58.222 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-05-08T22:37:58.223 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-05-08T22:37:58.224 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-05-08T22:37:58.224 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-05-08T22:37:58.224 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-05-08T22:37:58.224 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-05-08T22:37:58.224 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-05-08T22:37:58.224 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-05-08T22:37:58.234 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-05-08T22:37:58.234 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-05-08T22:37:58.234 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-05-08T22:37:58.234 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-05-08T22:37:58.234 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-05-08T22:37:58.234 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-05-08T22:37:58.234 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-05-08T22:37:58.234 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-05-08T22:37:58.234 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-05-08T22:37:58.235 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-05-08T22:37:58.236 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-05-08T22:37:58.237 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-05-08T22:37:58.238 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-05-08T22:37:58.239 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-05-08T22:37:58.256 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-05-08T22:37:58.256 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-05-08T22:37:58.256 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-05-08T22:37:58.256 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-05-08T22:37:58.256 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-05-08T22:37:58.256 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-05-08T22:37:58.256 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-05-08T22:37:58.256 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-05-08T22:37:58.257 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-05-08T22:37:58.258 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-05-08T22:37:58.259 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-05-08T22:37:58.260 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-05-08T22:37:58.261 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-05-08T22:37:58.267 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-05-08T22:37:58.267 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-05-08T22:37:58.267 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-05-08T22:37:58.267 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-05-08T22:37:58.267 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-05-08T22:37:58.267 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-05-08T22:37:58.267 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-05-08T22:37:58.267 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-05-08T22:37:58.267 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-05-08T22:37:58.267 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-05-08T22:37:58.268 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-05-08T22:37:58.269 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-05-08T22:37:58.270 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-05-08T22:37:58.271 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-05-08T22:37:58.272 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-05-08T22:37:58.280 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-05-08T22:37:58.280 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-05-08T22:37:58.281 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-05-08T22:37:58.282 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-05-08T22:37:58.283 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-05-08T22:37:58.284 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-05-08T22:37:58.285 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-05-08T22:37:58.286 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-05-08T22:37:58.286 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-05-08T22:37:58.286 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-05-08T22:37:58.286 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-05-08T22:37:58.286 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-05-08T22:37:58.286 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-05-08T22:37:58.286 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-05-08T22:37:58.286 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-05-08T22:37:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-05-08T22:37:58.302 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-05-08T22:37:58.303 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-05-08T22:37:58.304 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-05-08T22:37:58.304 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-05-08T22:37:58.304 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-05-08T22:37:58.304 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-05-08T22:37:58.304 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-05-08T22:37:58.304 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-05-08T22:37:58.304 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-05-08T22:37:58.304 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-05-08T22:37:58.305 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-05-08T22:37:58.306 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-05-08T22:37:58.307 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-05-08T22:37:58.307 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-05-08T22:37:58.307 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-05-08T22:37:58.324 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-05-08T22:37:58.324 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-05-08T22:37:58.324 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-05-08T22:37:58.325 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-05-08T22:37:58.326 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-05-08T22:37:58.327 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-05-08T22:37:58.328 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-05-08T22:37:58.329 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-05-08T22:37:58.330 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-05-08T22:37:58.330 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-05-08T22:37:58.330 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-05-08T22:37:58.330 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-05-08T22:37:58.330 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-05-08T22:37:58.330 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-05-08T22:37:58.336 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-05-08T22:37:58.336 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-05-08T22:37:58.336 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-05-08T22:37:58.336 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-05-08T22:37:58.336 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-05-08T22:37:58.337 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-05-08T22:37:58.338 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-05-08T22:37:58.339 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-05-08T22:37:58.340 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-05-08T22:37:58.340 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-05-08T22:37:58.340 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-05-08T22:37:58.340 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-05-08T22:37:58.340 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-05-08T22:37:58.340 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-05-08T22:37:58.340 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-05-08T22:37:58.340 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-05-08T22:37:58.340 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-05-08T22:37:58.340 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-05-08T22:37:58.340 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-05-08T22:37:58.341 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-05-08T22:37:58.342 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-05-08T22:37:58.342 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-05-08T22:37:58.342 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-05-08T22:37:58.342 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-05-08T22:37:58.342 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-05-08T22:37:58.342 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-05-08T22:37:58.342 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-05-08T22:37:58.342 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-05-08T22:37:58.342 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-05-08T22:37:58.342 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-05-08T22:37:58.348 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-05-08T22:37:58.348 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-05-08T22:37:58.348 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-05-08T22:37:58.348 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-05-08T22:37:58.348 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-05-08T22:37:58.348 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-05-08T22:37:58.349 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-05-08T22:37:58.350 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-05-08T22:37:58.351 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-05-08T22:37:58.352 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-05-08T22:37:58.353 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-05-08T22:37:58.354 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-05-08T22:37:58.354 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-05-08T22:37:58.354 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-05-08T22:37:58.367 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-05-08T22:37:58.367 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-05-08T22:37:58.367 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-05-08T22:37:58.367 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-05-08T22:37:58.367 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-05-08T22:37:58.368 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-05-08T22:37:58.369 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-05-08T22:37:58.370 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-05-08T22:37:58.371 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-05-08T22:37:58.372 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-05-08T22:37:58.373 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-05-08T22:37:58.373 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-05-08T22:37:58.373 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-05-08T22:37:58.373 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-05-08T22:37:58.373 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-05-08T22:37:58.387 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-05-08T22:37:58.387 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-05-08T22:37:58.387 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-05-08T22:37:58.387 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-05-08T22:37:58.387 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-05-08T22:37:58.388 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-05-08T22:37:58.389 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-05-08T22:37:58.390 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-05-08T22:37:58.391 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-05-08T22:37:58.392 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-05-08T22:37:58.393 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-05-08T22:37:58.393 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-05-08T22:37:58.393 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-05-08T22:37:58.393 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-05-08T22:37:58.393 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-05-08T22:37:58.393 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-05-08T22:37:58.393 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-05-08T22:37:58.397 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-05-08T22:37:58.397 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-05-08T22:37:58.397 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-05-08T22:37:58.397 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-05-08T22:37:58.397 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-05-08T22:37:58.397 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-05-08T22:37:58.397 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-05-08T22:37:58.397 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-05-08T22:37:58.397 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-05-08T22:37:58.398 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-05-08T22:37:58.399 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-05-08T22:37:58.400 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-05-08T22:37:58.401 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-05-08T22:37:58.402 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-05-08T22:37:58.495 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-05-08T22:37:58.495 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-05-08T22:37:58.495 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-05-08T22:37:58.496 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-05-08T22:37:58.497 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-05-08T22:37:58.498 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-05-08T22:37:58.499 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-05-08T22:37:58.499 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-05-08T22:37:58.499 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-05-08T22:37:58.499 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-05-08T22:37:58.499 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-05-08T22:37:58.500 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-05-08T22:37:58.501 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-05-08T22:37:58.502 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-05-08T22:37:58.502 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-05-08T22:37:58.502 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-05-08T22:37:58.502 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-05-08T22:37:58.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-05-08T22:37:58.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-05-08T22:37:58.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-05-08T22:37:58.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-05-08T22:37:58.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-05-08T22:37:58.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-05-08T22:37:59.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-05-08T22:37:59.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-05-08T22:37:59.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-05-08T22:37:59.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-05-08T22:37:59.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-05-08T22:37:59.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-05-08T22:37:59.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-05-08T22:37:59.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-05-08T22:37:59.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-05-08T22:37:59.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-05-08T22:37:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-05-08T22:37:59.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-05-08T22:37:59.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-05-08T22:37:59.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-05-08T22:37:59.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-05-08T22:37:59.209 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-05-08T22:37:59.209 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-05-08T22:37:59.209 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-05-08T22:37:59.209 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-05-08T22:37:59.209 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-05-08T22:37:59.209 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-05-08T22:37:59.209 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-05-08T22:37:59.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-05-08T22:37:59.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-05-08T22:37:59.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-05-08T22:37:59.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-05-08T22:37:59.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-05-08T22:37:59.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-05-08T22:37:59.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-05-08T22:37:59.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-05-08T22:37:59.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-05-08T22:37:59.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-05-08T22:37:59.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-05-08T22:37:59.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-05-08T22:37:59.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-05-08T22:37:59.558 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-05-08T22:37:59.559 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-05-08T22:37:59.559 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-05-08T22:37:59.559 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-05-08T22:37:59.559 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-05-08T22:37:59.559 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-05-08T22:37:59.559 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-05-08T22:37:59.559 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-05-08T22:37:59.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:37:59 smithi066 ceph-mon[46830]: pgmap v308: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.07k op/s 2024-05-08T22:37:59.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-05-08T22:37:59.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-05-08T22:37:59.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-05-08T22:37:59.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-05-08T22:37:59.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-05-08T22:37:59.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-05-08T22:37:59.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-05-08T22:37:59.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-05-08T22:37:59.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-05-08T22:37:59.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-05-08T22:37:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-05-08T22:37:59.886 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-05-08T22:37:59.887 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-05-08T22:37:59.887 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-05-08T22:37:59.887 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-05-08T22:37:59.887 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-05-08T22:37:59.887 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-05-08T22:37:59.887 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-05-08T22:37:59.887 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-05-08T22:37:59.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:37:59 smithi079 ceph-mon[46738]: pgmap v308: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.07k op/s 2024-05-08T22:37:59.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:37:59 smithi062 ceph-mon[42571]: pgmap v308: 97 pgs: 97 active+clean; 17 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.07k op/s 2024-05-08T22:38:00.249 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-05-08T22:38:00.249 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-05-08T22:38:00.250 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-05-08T22:38:00.251 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-05-08T22:38:00.252 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-05-08T22:38:00.253 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-05-08T22:38:00.254 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-05-08T22:38:00.255 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-05-08T22:38:00.255 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-05-08T22:38:00.255 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-05-08T22:38:00.255 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-05-08T22:38:00.255 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-05-08T22:38:00.255 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-05-08T22:38:00.255 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-05-08T22:38:00.255 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-05-08T22:38:00.255 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-05-08T22:38:00.255 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-05-08T22:38:00.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-05-08T22:38:00.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-05-08T22:38:00.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-05-08T22:38:00.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-05-08T22:38:00.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-05-08T22:38:00.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-05-08T22:38:00.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-05-08T22:38:00.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-05-08T22:38:00.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-05-08T22:38:00.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-05-08T22:38:00.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-05-08T22:38:00.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-05-08T22:38:00.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-05-08T22:38:00.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-05-08T22:38:00.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-05-08T22:38:00.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-05-08T22:38:00.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-05-08T22:38:00.849 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-05-08T22:38:00.850 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-05-08T22:38:00.850 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-05-08T22:38:00.850 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-05-08T22:38:00.850 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-05-08T22:38:00.850 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-05-08T22:38:00.850 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-05-08T22:38:00.850 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-05-08T22:38:00.850 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-05-08T22:38:00.850 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-05-08T22:38:00.850 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-05-08T22:38:00.851 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-05-08T22:38:00.852 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-05-08T22:38:00.853 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-05-08T22:38:00.854 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-05-08T22:38:00.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-05-08T22:38:00.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-05-08T22:38:00.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-05-08T22:38:00.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-05-08T22:38:00.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-05-08T22:38:00.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-05-08T22:38:01.159 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-05-08T22:38:01.159 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-05-08T22:38:01.159 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-05-08T22:38:01.160 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-05-08T22:38:01.161 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-05-08T22:38:01.162 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-05-08T22:38:01.163 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-05-08T22:38:01.164 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-05-08T22:38:01.165 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-05-08T22:38:01.165 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-05-08T22:38:01.165 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-05-08T22:38:01.547 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-05-08T22:38:01.548 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-05-08T22:38:01.549 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-05-08T22:38:01.550 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-05-08T22:38:01.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-05-08T22:38:01.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-05-08T22:38:01.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-05-08T22:38:01.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:01 smithi066 ceph-mon[46830]: pgmap v309: 97 pgs: 97 active+clean; 129 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.1 MiB/s wr, 4.04k op/s 2024-05-08T22:38:01.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-05-08T22:38:01.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-05-08T22:38:01.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-05-08T22:38:01.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-05-08T22:38:01.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-05-08T22:38:01.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-05-08T22:38:01.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-05-08T22:38:01.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-05-08T22:38:01.873 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-05-08T22:38:01.874 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-05-08T22:38:01.875 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-05-08T22:38:01.876 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-05-08T22:38:01.877 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-05-08T22:38:01.877 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-05-08T22:38:01.877 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-05-08T22:38:01.877 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-05-08T22:38:01.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:01 smithi079 ceph-mon[46738]: pgmap v309: 97 pgs: 97 active+clean; 129 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.1 MiB/s wr, 4.04k op/s 2024-05-08T22:38:01.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:01 smithi062 ceph-mon[42571]: pgmap v309: 97 pgs: 97 active+clean; 129 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.1 MiB/s wr, 4.04k op/s 2024-05-08T22:38:02.198 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-05-08T22:38:02.198 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-05-08T22:38:02.198 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-05-08T22:38:02.198 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-05-08T22:38:02.198 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-05-08T22:38:02.198 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-05-08T22:38:02.198 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-05-08T22:38:02.198 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-05-08T22:38:02.199 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-05-08T22:38:02.200 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-05-08T22:38:02.201 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-05-08T22:38:02.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-05-08T22:38:02.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-05-08T22:38:02.526 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-05-08T22:38:02.526 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-05-08T22:38:02.526 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-05-08T22:38:02.526 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-05-08T22:38:02.526 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-05-08T22:38:02.526 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-05-08T22:38:02.526 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-05-08T22:38:02.526 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-05-08T22:38:02.527 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-05-08T22:38:02.528 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-05-08T22:38:02.529 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-05-08T22:38:02.530 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-05-08T22:38:02.530 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-05-08T22:38:02.530 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-05-08T22:38:02.530 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-05-08T22:38:02.530 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-05-08T22:38:02.530 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-05-08T22:38:02.530 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-05-08T22:38:02.530 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-05-08T22:38:02.530 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-05-08T22:38:02.530 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-05-08T22:38:02.531 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-05-08T22:38:02.532 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-05-08T22:38:02.532 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-05-08T22:38:02.532 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-05-08T22:38:02.532 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-05-08T22:38:02.532 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-05-08T22:38:02.866 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-05-08T22:38:02.866 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-05-08T22:38:02.866 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-05-08T22:38:02.866 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-05-08T22:38:02.866 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-05-08T22:38:02.866 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-05-08T22:38:02.866 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-05-08T22:38:02.866 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-05-08T22:38:02.866 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-05-08T22:38:02.866 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-05-08T22:38:02.867 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-05-08T22:38:02.868 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-05-08T22:38:02.869 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-05-08T22:38:02.870 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-05-08T22:38:02.870 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-05-08T22:38:02.870 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-05-08T22:38:02.870 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-05-08T22:38:02.870 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-05-08T22:38:02.870 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-05-08T22:38:02.870 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-05-08T22:38:02.870 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-05-08T22:38:02.870 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-05-08T22:38:02.871 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-05-08T22:38:02.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-05-08T22:38:02.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-05-08T22:38:02.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-05-08T22:38:02.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-05-08T22:38:02.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-05-08T22:38:02.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-05-08T22:38:02.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-05-08T22:38:02.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-05-08T22:38:02.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-05-08T22:38:02.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-05-08T22:38:02.872 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-05-08T22:38:03.178 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-05-08T22:38:03.178 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-05-08T22:38:03.178 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-05-08T22:38:03.178 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-05-08T22:38:03.178 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-05-08T22:38:03.179 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-05-08T22:38:03.180 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-05-08T22:38:03.181 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-05-08T22:38:03.182 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-05-08T22:38:03.182 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-05-08T22:38:03.182 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-05-08T22:38:03.182 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-05-08T22:38:03.182 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-05-08T22:38:03.182 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-05-08T22:38:03.182 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-05-08T22:38:03.182 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-05-08T22:38:03.182 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-05-08T22:38:03.182 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-05-08T22:38:03.182 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-05-08T22:38:03.183 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-05-08T22:38:03.184 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-05-08T22:38:03.184 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-05-08T22:38:03.184 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-05-08T22:38:03.184 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-05-08T22:38:03.184 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-05-08T22:38:03.184 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-05-08T22:38:03.184 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-05-08T22:38:03.184 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-05-08T22:38:03.184 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-05-08T22:38:03.184 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-05-08T22:38:03.508 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-05-08T22:38:03.509 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-05-08T22:38:03.510 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-05-08T22:38:03.511 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-05-08T22:38:03.512 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-05-08T22:38:03.513 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-05-08T22:38:03.514 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-05-08T22:38:03.514 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-05-08T22:38:03.514 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-05-08T22:38:03.514 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-05-08T22:38:03.514 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-05-08T22:38:03.514 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-05-08T22:38:03.514 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-05-08T22:38:03.514 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-05-08T22:38:03.514 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-05-08T22:38:03.514 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-05-08T22:38:03.514 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-05-08T22:38:03.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:03 smithi066 ceph-mon[46830]: pgmap v310: 97 pgs: 97 active+clean; 577 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 3.12k op/s 2024-05-08T22:38:03.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-05-08T22:38:03.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-05-08T22:38:03.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-05-08T22:38:03.855 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-05-08T22:38:03.856 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-05-08T22:38:03.857 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-05-08T22:38:03.858 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-05-08T22:38:03.859 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-05-08T22:38:03.860 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-05-08T22:38:03.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-05-08T22:38:03.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-05-08T22:38:03.861 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-05-08T22:38:03.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:03 smithi079 ceph-mon[46738]: pgmap v310: 97 pgs: 97 active+clean; 577 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 3.12k op/s 2024-05-08T22:38:03.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:03 smithi062 ceph-mon[42571]: pgmap v310: 97 pgs: 97 active+clean; 577 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 3.12k op/s 2024-05-08T22:38:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.932+0000 7f53fa0b3700 1 -- 172.21.15.62:0/3396854086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4106b80 msgr2=0x7f53f4108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.932+0000 7f53fa0b3700 1 --2- 172.21.15.62:0/3396854086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4106b80 0x7f53f4108ff0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009990 tx=0x7f53ec009ca0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.932+0000 7f53fa0b3700 1 -- 172.21.15.62:0/3396854086 shutdown_connections 2024-05-08T22:38:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.932+0000 7f53fa0b3700 1 --2- 172.21.15.62:0/3396854086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4106b80 0x7f53f4108ff0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.932+0000 7f53fa0b3700 1 --2- 172.21.15.62:0/3396854086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f4101920 0x7f53f4103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.932+0000 7f53fa0b3700 1 --2- 172.21.15.62:0/3396854086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53f4104250 0x7f53f4106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.932+0000 7f53fa0b3700 1 -- 172.21.15.62:0/3396854086 >> 172.21.15.62:0/3396854086 conn(0x7f53f40fb310 msgr2=0x7f53f40fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.932+0000 7f53fa0b3700 1 -- 172.21.15.62:0/3396854086 shutdown_connections 2024-05-08T22:38:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.932+0000 7f53fa0b3700 1 -- 172.21.15.62:0/3396854086 wait complete. 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.933+0000 7f53fa0b3700 1 Processor -- start 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.933+0000 7f53fa0b3700 1 -- start start 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.933+0000 7f53fa0b3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4101920 0x7f53f4196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.933+0000 7f53fa0b3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53f4104250 0x7f53f4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.933+0000 7f53fa0b3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f4106b80 0x7f53f419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.933+0000 7f53fa0b3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f419e840 con 0x7f53f4101920 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.933+0000 7f53fa0b3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f419e9b0 con 0x7f53f4106b80 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.933+0000 7f53fa0b3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f419eb70 con 0x7f53f4104250 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4101920 0x7f53f4196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53f4104250 0x7f53f4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4101920 0x7f53f4196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49508/0 (socket says 172.21.15.62:49508) 2024-05-08T22:38:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f37fe700 1 -- 172.21.15.62:0/1744417834 learned_addr learned my addr 172.21.15.62:0/1744417834 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f3fff700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f4106b80 0x7f53f419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f37fe700 1 -- 172.21.15.62:0/1744417834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53f4104250 msgr2=0x7f53f4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f37fe700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53f4104250 0x7f53f4197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f37fe700 1 -- 172.21.15.62:0/1744417834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f4106b80 msgr2=0x7f53f419e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f37fe700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f4106b80 0x7f53f419e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f37fe700 1 -- 172.21.15.62:0/1744417834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0095f0 con 0x7f53f4101920 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f3fff700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f4106b80 0x7f53f419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f37fe700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4101920 0x7f53f4196d50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f53e40060b0 tx=0x7f53e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f0ff9700 1 -- 172.21.15.62:0/1744417834 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e400dd20 con 0x7f53f4101920 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f0ff9700 1 -- 172.21.15.62:0/1744417834 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e400f040 con 0x7f53f4101920 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53fa0b3700 1 -- 172.21.15.62:0/1744417834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f419ee50 con 0x7f53f4101920 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53f0ff9700 1 -- 172.21.15.62:0/1744417834 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4014600 con 0x7f53f4101920 2024-05-08T22:38:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.934+0000 7f53fa0b3700 1 -- 172.21.15.62:0/1744417834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f419f270 con 0x7f53f4101920 2024-05-08T22:38:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.936+0000 7f53f0ff9700 1 -- 172.21.15.62:0/1744417834 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f53e4015400 con 0x7f53f4101920 2024-05-08T22:38:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.937+0000 7f53f0ff9700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f53e00a1110 0x7f53e00a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.937+0000 7f53f0ff9700 1 -- 172.21.15.62:0/1744417834 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f53e40cc1d0 con 0x7f53f4101920 2024-05-08T22:38:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.937+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f53e00a1110 0x7f53e00a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.938+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f53e00a1110 0x7f53e00a35d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009c60 tx=0x7f53dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.938+0000 7f540626db80 1 Processor -- start 2024-05-08T22:38:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.938+0000 7f540626db80 1 -- start start 2024-05-08T22:38:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.938+0000 7f540626db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 0x557e03f44ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.938+0000 7f540626db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f453e0 0x557e03f45cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.938+0000 7f540626db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f462d0 0x557e03f46750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.938+0000 7f540626db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03e9c0f0 con 0x557e03f44a80 2024-05-08T22:38:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.938+0000 7f540626db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03e99ab0 con 0x557e03f453e0 2024-05-08T22:38:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.938+0000 7f540626db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03e953a0 con 0x557e03f462d0 2024-05-08T22:38:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.938+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 0x557e03f44ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f453e0 0x557e03f45cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 0x557e03f44ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49520/0 (socket says 172.21.15.62:49520) 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53f37fe700 1 -- 172.21.15.62:0/2880158159 learned_addr learned my addr 172.21.15.62:0/2880158159 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53f3fff700 1 --2- 172.21.15.62:0/2880158159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f462d0 0x557e03f46750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53f2ffd700 1 -- 172.21.15.62:0/2880158159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f462d0 msgr2=0x557e03f46750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/2880158159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f462d0 0x557e03f46750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53f2ffd700 1 -- 172.21.15.62:0/2880158159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 msgr2=0x557e03f44ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/2880158159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 0x557e03f44ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53f2ffd700 1 -- 172.21.15.62:0/2880158159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e03d16350 con 0x557e03f453e0 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/2880158159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f453e0 0x557e03f45cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00e3b0 tx=0x7f53dc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28519f18ec0274e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53e9ffb700 1 -- 172.21.15.62:0/2880158159 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00ed50 con 0x557e03f453e0 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53e9ffb700 1 -- 172.21.15.62:0/2880158159 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53dc021070 con 0x557e03f453e0 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.939+0000 7f53e9ffb700 1 -- 172.21.15.62:0/2880158159 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc003ea0 con 0x557e03f453e0 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.940+0000 7f540626db80 1 -- 172.21.15.62:0/2880158159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f453e0 msgr2=0x557e03f45cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.940+0000 7f540626db80 1 --2- 172.21.15.62:0/2880158159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f453e0 0x557e03f45cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00e3b0 tx=0x7f53dc014000 comp rx=0 tx=0).stop 2024-05-08T22:38:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.940+0000 7f540626db80 1 -- 172.21.15.62:0/2880158159 shutdown_connections 2024-05-08T22:38:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.940+0000 7f540626db80 1 --2- 172.21.15.62:0/2880158159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f462d0 0x557e03f46750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.940+0000 7f540626db80 1 --2- 172.21.15.62:0/2880158159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 0x557e03f44ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.940+0000 7f540626db80 1 --2- 172.21.15.62:0/2880158159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f453e0 0x557e03f45cf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.940+0000 7f540626db80 1 -- 172.21.15.62:0/2880158159 >> 172.21.15.62:0/2880158159 conn(0x557e03ea31b0 msgr2=0x557e03f41440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.940+0000 7f540626db80 1 -- 172.21.15.62:0/2880158159 shutdown_connections 2024-05-08T22:38:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.940+0000 7f540626db80 1 -- 172.21.15.62:0/2880158159 wait complete. 2024-05-08T22:38:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.940+0000 7f540626db80 1 -- start start 2024-05-08T22:38:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f540626db80 1 Processor -- start 2024-05-08T22:38:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f540626db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 0x557e03f4e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f540626db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f453e0 0x557e03f4edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f540626db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f462d0 0x557e03f55e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f540626db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03e31500 con 0x557e03f44a80 2024-05-08T22:38:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f540626db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03e311a0 con 0x557e03f462d0 2024-05-08T22:38:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f540626db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03d18640 con 0x557e03f453e0 2024-05-08T22:38:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f540626db80 10 client.0 _ll_register_callbacks cb 0x557e03e3aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 0x557e03f4e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 0x557e03f4e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49528/0 (socket says 172.21.15.62:49528) 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f37fe700 1 -- 172.21.15.62:0/2416722765 learned_addr learned my addr 172.21.15.62:0/2416722765 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f3fff700 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f462d0 0x557e03f55e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f453e0 0x557e03f4edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f37fe700 1 -- 172.21.15.62:0/2416722765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f453e0 msgr2=0x557e03f4edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f37fe700 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f453e0 0x557e03f4edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f37fe700 1 -- 172.21.15.62:0/2416722765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f462d0 msgr2=0x557e03f55e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f37fe700 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f462d0 0x557e03f55e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f37fe700 1 -- 172.21.15.62:0/2416722765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53dc017070 con 0x557e03f44a80 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f453e0 0x557e03f4edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53f37fe700 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 0x557e03f4e880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f53e4095ae0 tx=0x7f53e4099400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c3e77ad215470c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.941+0000 7f53d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-05-08T22:38:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f53d2ffd700 1 -- 172.21.15.62:0/2416722765 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e40b90b0 con 0x557e03f44a80 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f53d2ffd700 1 -- 172.21.15.62:0/2416722765 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f53e4099d30 con 0x557e03f44a80 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f540626db80 1 -- 172.21.15.62:0/2416722765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e03dee210 con 0x557e03f44a80 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f53d2ffd700 1 -- 172.21.15.62:0/2416722765 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e4099630 con 0x557e03f44a80 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f540626db80 10 client.15498 fetch_fsmap learned FSMap version 29 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f540626db80 1 -- 172.21.15.62:0/2416722765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557e03cc2b00 con 0x557e03f44a80 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f53d2ffd700 1 -- 172.21.15.62:0/2416722765 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e03dee210 con 0x557e03f44a80 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f53d2ffd700 1 -- 172.21.15.62:0/2416722765 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f53e40840b0 con 0x557e03f44a80 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f540626db80 10 client.15498 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f540626db80 10 client.15498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f540626db80 20 client.15498 populate_metadata read hostname 'smithi062' 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f540626db80 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x557e03f5a710 0x557e03f5cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f540626db80 4 client.15498 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.942+0000 7f540626db80 1 -- 172.21.15.62:0/2416722765 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e03f147b0 con 0x557e03f5a710 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.943+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x557e03f5a710 0x557e03f5cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.943+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x557e03f5a710 0x557e03f5cb30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e22677e50e90d39 server_cookie=8c584f09c1fc9ab in_seq=0 out_seq=0 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.943+0000 7f53d2ffd700 10 client.15498 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.943+0000 7f53d2ffd700 1 -- 172.21.15.62:0/2416722765 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e03f147b0 con 0x557e03f5a710 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.943+0000 7f53d2ffd700 10 client.15498 handle_command_reply: tid=0 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 1 client.15498 shutdown 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 1 -- 172.21.15.62:0/2416722765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 msgr2=0x557e03f4e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 0x557e03f4e880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f53e4095ae0 tx=0x7f53e4099400 comp rx=0 tx=0).stop 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 1 -- 172.21.15.62:0/2416722765 shutdown_connections 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x557e03f5a710 0x557e03f5cb30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f462d0 0x557e03f55e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f44a80 0x557e03f4e880 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f53d2ffd700 0 client.15498 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 1 --2- 172.21.15.62:0/2416722765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f453e0 0x557e03f4edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 1 -- 172.21.15.62:0/2416722765 >> 172.21.15.62:0/2416722765 conn(0x557e03ea31b0 msgr2=0x557e03f3f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 1 -- 172.21.15.62:0/2416722765 shutdown_connections 2024-05-08T22:38:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 1 -- 172.21.15.62:0/2416722765 wait complete. 2024-05-08T22:38:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.944+0000 7f540626db80 20 client.15498 trim_cache size 0 max 16384 2024-05-08T22:38:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f540626db80 1 Processor -- start 2024-05-08T22:38:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f540626db80 1 -- start start 2024-05-08T22:38:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f540626db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f44a80 0x557e03f44e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f540626db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 0x557e03f55dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f540626db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f462d0 0x557e03f466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f540626db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03f457c0 con 0x557e03f453e0 2024-05-08T22:38:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f540626db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03f61d20 con 0x557e03f462d0 2024-05-08T22:38:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f540626db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03f45ae0 con 0x557e03f44a80 2024-05-08T22:38:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 0x557e03f55dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f44a80 0x557e03f44e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f53f3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f462d0 0x557e03f466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f44a80 0x557e03f44e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.62:43794/0 (socket says 172.21.15.62:43794) 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 0x557e03f55dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49538/0 (socket says 172.21.15.62:49538) 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f53f3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f462d0 0x557e03f466b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.62:48048/0 (socket says 172.21.15.62:48048) 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.959+0000 7f53f37fe700 1 -- 172.21.15.62:0/1987328741 learned_addr learned my addr 172.21.15.62:0/1987328741 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.960+0000 7f53f37fe700 1 -- 172.21.15.62:0/1987328741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f462d0 msgr2=0x557e03f466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.960+0000 7f53f37fe700 1 --2- 172.21.15.62:0/1987328741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f462d0 0x557e03f466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.960+0000 7f53f37fe700 1 -- 172.21.15.62:0/1987328741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 msgr2=0x557e03f55dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.960+0000 7f53f37fe700 1 --2- 172.21.15.62:0/1987328741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 0x557e03f55dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.960+0000 7f53f37fe700 1 -- 172.21.15.62:0/1987328741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e03e5b500 con 0x557e03f44a80 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.960+0000 7f53f37fe700 1 --2- 172.21.15.62:0/1987328741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f44a80 0x557e03f44e60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53e40ca050 tx=0x7f53e4097be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec361aa9774c35ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.960+0000 7f53ea7fc700 1 -- 172.21.15.62:0/1987328741 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e40a5070 con 0x557e03f44a80 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.960+0000 7f53ea7fc700 1 -- 172.21.15.62:0/1987328741 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e4098b20 con 0x557e03f44a80 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.960+0000 7f53ea7fc700 1 -- 172.21.15.62:0/1987328741 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e409d460 con 0x557e03f44a80 2024-05-08T22:38:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 -- 172.21.15.62:0/1987328741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f44a80 msgr2=0x557e03f44e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 --2- 172.21.15.62:0/1987328741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f44a80 0x557e03f44e60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53e40ca050 tx=0x7f53e4097be0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 -- 172.21.15.62:0/1987328741 shutdown_connections 2024-05-08T22:38:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 --2- 172.21.15.62:0/1987328741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f462d0 0x557e03f466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 --2- 172.21.15.62:0/1987328741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f44a80 0x557e03f44e60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 --2- 172.21.15.62:0/1987328741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 0x557e03f55dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 -- 172.21.15.62:0/1987328741 >> 172.21.15.62:0/1987328741 conn(0x557e03ea31b0 msgr2=0x557e03f4de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 -- 172.21.15.62:0/1987328741 shutdown_connections 2024-05-08T22:38:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 -- 172.21.15.62:0/1987328741 wait complete. 2024-05-08T22:38:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 -- start start 2024-05-08T22:38:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 Processor -- start 2024-05-08T22:38:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f44a80 0x557e03f53320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 0x557e03f53860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f462d0 0x557e03f54870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03f54db0 con 0x557e03f453e0 2024-05-08T22:38:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03f54f20 con 0x557e03f44a80 2024-05-08T22:38:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.961+0000 7f540626db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e03f550e0 con 0x557e03f462d0 2024-05-08T22:38:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f540626db80 10 client.0 _ll_register_callbacks cb 0x557e03e3aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T22:38:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 0x557e03f53860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f44a80 0x557e03f53320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 0x557e03f53860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49552/0 (socket says 172.21.15.62:49552) 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53f2ffd700 1 -- 172.21.15.62:0/229436898 learned_addr learned my addr 172.21.15.62:0/229436898 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53f3fff700 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f462d0 0x557e03f54870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53f37fe700 1 -- 172.21.15.62:0/229436898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f462d0 msgr2=0x557e03f54870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53f37fe700 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f462d0 0x557e03f54870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53f37fe700 1 -- 172.21.15.62:0/229436898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 msgr2=0x557e03f53860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53f37fe700 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 0x557e03f53860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53f37fe700 1 -- 172.21.15.62:0/229436898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e03cc2b00 con 0x557e03f44a80 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53f37fe700 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f44a80 0x557e03f53320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f53e4099330 tx=0x7f53e4098ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ad099d47f0c439 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53d2ffd700 1 -- 172.21.15.62:0/229436898 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e40a5060 con 0x557e03f44a80 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53d2ffd700 1 -- 172.21.15.62:0/229436898 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f53e40b90b0 con 0x557e03f44a80 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f540626db80 1 -- 172.21.15.62:0/229436898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e03e8de60 con 0x557e03f44a80 2024-05-08T22:38:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f53d2ffd700 1 -- 172.21.15.62:0/229436898 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e4098b20 con 0x557e03f44a80 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f540626db80 10 client.25202 fetch_fsmap learned FSMap version 29 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.962+0000 7f540626db80 1 -- 172.21.15.62:0/229436898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557e03d1ac70 con 0x557e03f44a80 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.963+0000 7f53d2ffd700 1 -- 172.21.15.62:0/229436898 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e03e8de60 con 0x557e03f44a80 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.963+0000 7f53d2ffd700 1 -- 172.21.15.62:0/229436898 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f53e4095420 con 0x557e03f44a80 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.963+0000 7f540626db80 10 client.25202 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.963+0000 7f540626db80 10 client.25202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.963+0000 7f540626db80 20 client.25202 populate_metadata read hostname 'smithi062' 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.963+0000 7f540626db80 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x557e03fb0ed0 0x557e03fb32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.963+0000 7f540626db80 4 client.25202 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.963+0000 7f540626db80 1 -- 172.21.15.62:0/229436898 --> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557e03ce3c90 con 0x557e03fb0ed0 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.963+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x557e03fb0ed0 0x557e03fb32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.963+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x557e03fb0ed0 0x557e03fb32f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ded23b9ffd6ef4d server_cookie=6123202f0da10678 in_seq=0 out_seq=0 2024-05-08T22:38:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.963+0000 7f53d2ffd700 10 client.25202 ms_handle_connect on v2:172.21.15.66:6838/3474672912 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f53d2ffd700 1 -- 172.21.15.62:0/229436898 <== mds.0 v2:172.21.15.66:6838/3474672912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557e03ce3c90 con 0x557e03fb0ed0 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f53d2ffd700 10 client.25202 handle_command_reply: tid=0 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 1 client.25202 shutdown 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 1 -- 172.21.15.62:0/229436898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f44a80 msgr2=0x557e03f53320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f44a80 0x557e03f53320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f53e4099330 tx=0x7f53e4098ee0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 1 -- 172.21.15.62:0/229436898 shutdown_connections 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.66:6838/3474672912,v1:172.21.15.66:6839/3474672912] conn(0x557e03fb0ed0 0x557e03fb32f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557e03f462d0 0x557e03f54870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e03f44a80 0x557e03f53320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f53d2ffd700 0 client.25202 ms_handle_reset on v2:172.21.15.66:6838/3474672912 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 1 --2- 172.21.15.62:0/229436898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e03f453e0 0x557e03f53860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 1 -- 172.21.15.62:0/229436898 >> 172.21.15.62:0/229436898 conn(0x557e03ea31b0 msgr2=0x557e03f61490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 1 -- 172.21.15.62:0/229436898 shutdown_connections 2024-05-08T22:38:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 1 -- 172.21.15.62:0/229436898 wait complete. 2024-05-08T22:38:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.969+0000 7f540626db80 20 client.25202 trim_cache size 0 max 16384 2024-05-08T22:38:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2024-05-08T22:38:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2024-05-08T22:38:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2024-05-08T22:38:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 -- 172.21.15.62:0/1744417834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f53e00a1110 msgr2=0x7f53e00a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f53e00a1110 0x7f53e00a35d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009c60 tx=0x7f53dc009260 comp rx=0 tx=0).stop 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 -- 172.21.15.62:0/1744417834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4101920 msgr2=0x7f53f4196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4101920 0x7f53f4196d50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f53e40060b0 tx=0x7f53e400d3b0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 -- 172.21.15.62:0/1744417834 shutdown_connections 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f53e00a1110 0x7f53e00a35d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f4106b80 0x7f53f419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4101920 0x7f53f4196d50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 --2- 172.21.15.62:0/1744417834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53f4104250 0x7f53f4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 -- 172.21.15.62:0/1744417834 >> 172.21.15.62:0/1744417834 conn(0x7f53f40fb310 msgr2=0x7f53f4102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 -- 172.21.15.62:0/1744417834 shutdown_connections 2024-05-08T22:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2024-05-08T22:38:03.970+0000 7f53fa0b3700 1 -- 172.21.15.62:0/1744417834 wait complete. 2024-05-08T22:38:04.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-05-08T22:38:04.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-05-08T22:38:04.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-05-08T22:38:04.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-05-08T22:38:04.202 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-05-08T22:38:04.203 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-05-08T22:38:04.204 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-05-08T22:38:04.205 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-05-08T22:38:04.206 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-05-08T22:38:04.207 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-05-08T22:38:04.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-05-08T22:38:04.208 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-05-08T22:38:04.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-05-08T22:38:04.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-05-08T22:38:04.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-05-08T22:38:04.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-05-08T22:38:04.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-05-08T22:38:04.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-05-08T22:38:04.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-05-08T22:38:04.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-05-08T22:38:04.551 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-05-08T22:38:04.552 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-05-08T22:38:04.553 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-05-08T22:38:04.554 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-05-08T22:38:04.555 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-05-08T22:38:04.556 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-05-08T22:38:04.557 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-05-08T22:38:04.880 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-05-08T22:38:04.880 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-05-08T22:38:04.880 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-05-08T22:38:04.880 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-05-08T22:38:04.880 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-05-08T22:38:04.880 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-05-08T22:38:04.880 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-05-08T22:38:04.880 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-05-08T22:38:04.880 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-05-08T22:38:04.880 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-05-08T22:38:04.881 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-05-08T22:38:04.882 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-05-08T22:38:04.883 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-05-08T22:38:04.884 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-05-08T22:38:04.885 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-05-08T22:38:05.221 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-05-08T22:38:05.221 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-05-08T22:38:05.221 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-05-08T22:38:05.221 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-05-08T22:38:05.221 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-05-08T22:38:05.221 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-05-08T22:38:05.222 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-05-08T22:38:05.223 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-05-08T22:38:05.224 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-05-08T22:38:05.225 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-05-08T22:38:05.226 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-05-08T22:38:05.227 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-05-08T22:38:05.227 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-05-08T22:38:05.227 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-05-08T22:38:05.227 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-05-08T22:38:05.227 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-05-08T22:38:05.227 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-05-08T22:38:05.227 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-05-08T22:38:05.571 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-05-08T22:38:05.571 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-05-08T22:38:05.571 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-05-08T22:38:05.571 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-05-08T22:38:05.571 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-05-08T22:38:05.571 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-05-08T22:38:05.572 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-05-08T22:38:05.573 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-05-08T22:38:05.574 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-05-08T22:38:05.575 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-05-08T22:38:05.576 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-05-08T22:38:05.577 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-05-08T22:38:05.577 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-05-08T22:38:05.577 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-05-08T22:38:05.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:05 smithi066 ceph-mon[46830]: pgmap v311: 97 pgs: 97 active+clean; 802 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 3.14k op/s 2024-05-08T22:38:05.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:05 smithi079 ceph-mon[46738]: pgmap v311: 97 pgs: 97 active+clean; 802 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 3.14k op/s 2024-05-08T22:38:05.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-05-08T22:38:05.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-05-08T22:38:05.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-05-08T22:38:05.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-05-08T22:38:05.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-05-08T22:38:05.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-05-08T22:38:05.903 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-05-08T22:38:05.904 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-05-08T22:38:05.904 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-05-08T22:38:05.904 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-05-08T22:38:05.904 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-05-08T22:38:05.904 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-05-08T22:38:05.904 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-05-08T22:38:05.904 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-05-08T22:38:05.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:05 smithi062 ceph-mon[42571]: pgmap v311: 97 pgs: 97 active+clean; 802 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 3.14k op/s 2024-05-08T22:38:06.228 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-05-08T22:38:06.228 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-05-08T22:38:06.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-05-08T22:38:06.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-05-08T22:38:06.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-05-08T22:38:06.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-05-08T22:38:06.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-05-08T22:38:06.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-05-08T22:38:06.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-05-08T22:38:06.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-05-08T22:38:06.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-05-08T22:38:06.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-05-08T22:38:06.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-05-08T22:38:06.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-05-08T22:38:06.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-05-08T22:38:06.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-05-08T22:38:06.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-05-08T22:38:06.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-05-08T22:38:06.559 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-05-08T22:38:06.559 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-05-08T22:38:06.559 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-05-08T22:38:06.559 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-05-08T22:38:06.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-05-08T22:38:06.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-05-08T22:38:06.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-05-08T22:38:06.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-05-08T22:38:06.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-05-08T22:38:06.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-05-08T22:38:06.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-05-08T22:38:06.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-05-08T22:38:06.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-05-08T22:38:06.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-05-08T22:38:06.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-05-08T22:38:06.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-05-08T22:38:06.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-05-08T22:38:06.913 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-05-08T22:38:06.913 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-05-08T22:38:06.913 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-05-08T22:38:06.913 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-05-08T22:38:06.913 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-05-08T22:38:06.913 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-05-08T22:38:06.913 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-05-08T22:38:06.914 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-05-08T22:38:06.915 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-05-08T22:38:06.916 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-05-08T22:38:06.917 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-05-08T22:38:06.918 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-05-08T22:38:06.919 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-05-08T22:38:06.919 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-05-08T22:38:06.919 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-05-08T22:38:06.919 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-05-08T22:38:06.919 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-05-08T22:38:06.919 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-05-08T22:38:07.235 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-05-08T22:38:07.235 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-05-08T22:38:07.235 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-05-08T22:38:07.235 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-05-08T22:38:07.235 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-05-08T22:38:07.235 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-05-08T22:38:07.235 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-05-08T22:38:07.235 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-05-08T22:38:07.235 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-05-08T22:38:07.236 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-05-08T22:38:07.237 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-05-08T22:38:07.238 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-05-08T22:38:07.239 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-05-08T22:38:07.240 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-05-08T22:38:07.241 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-05-08T22:38:07.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-05-08T22:38:07.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-05-08T22:38:07.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-05-08T22:38:07.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-05-08T22:38:07.560 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-05-08T22:38:07.561 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-05-08T22:38:07.562 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-05-08T22:38:07.563 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-05-08T22:38:07.564 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-05-08T22:38:07.565 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-05-08T22:38:07.566 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-05-08T22:38:07.566 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-05-08T22:38:07.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:07 smithi066 ceph-mon[46830]: pgmap v312: 97 pgs: 97 active+clean; 1.1 GiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.2 MiB/s wr, 1.20k op/s 2024-05-08T22:38:07.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:07 smithi079 ceph-mon[46738]: pgmap v312: 97 pgs: 97 active+clean; 1.1 GiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.2 MiB/s wr, 1.20k op/s 2024-05-08T22:38:07.896 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-05-08T22:38:07.896 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-05-08T22:38:07.897 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-05-08T22:38:07.898 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-05-08T22:38:07.899 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-05-08T22:38:07.900 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-05-08T22:38:07.901 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-05-08T22:38:07.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-05-08T22:38:07.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-05-08T22:38:07.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-05-08T22:38:07.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-05-08T22:38:07.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-05-08T22:38:07.902 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-05-08T22:38:07.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:07 smithi062 ceph-mon[42571]: pgmap v312: 97 pgs: 97 active+clean; 1.1 GiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.2 MiB/s wr, 1.20k op/s 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-05-08T22:38:08.229 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-05-08T22:38:08.230 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-05-08T22:38:08.231 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-05-08T22:38:08.232 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-05-08T22:38:08.233 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-05-08T22:38:08.234 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-05-08T22:38:08.280 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-05-08T22:38:08.281 INFO:tasks.workunit.client.0.smithi062.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-05-08T22:38:08.282 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T22:38:08.282 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T22:38:08.315 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-05-08T22:38:08.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-08T22:38:08.615 DEBUG:teuthology.parallel:result is None 2024-05-08T22:38:08.616 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T22:38:08.651 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T22:38:08.652 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-08T22:38:08.662 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi062.front.sepia.ceph.com 2024-05-08T22:38:08.699 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-08T22:38:08.710 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:38:08.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.868+0000 7f02fcaee700 1 -- 172.21.15.62:0/3158059610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8106b60 msgr2=0x7f02f8108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:08.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.868+0000 7f02fcaee700 1 --2- 172.21.15.62:0/3158059610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8106b60 0x7f02f8108fd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f02f000b0a0 tx=0x7f02f000b3b0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.868+0000 7f02fcaee700 1 -- 172.21.15.62:0/3158059610 shutdown_connections 2024-05-08T22:38:08.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.868+0000 7f02fcaee700 1 --2- 172.21.15.62:0/3158059610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8106b60 0x7f02f8108fd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.868+0000 7f02fcaee700 1 --2- 172.21.15.62:0/3158059610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02f8101900 0x7f02f8103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.868+0000 7f02fcaee700 1 --2- 172.21.15.62:0/3158059610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02f8104230 0x7f02f8106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.868+0000 7f02fcaee700 1 -- 172.21.15.62:0/3158059610 >> 172.21.15.62:0/3158059610 conn(0x7f02f80fb2f0 msgr2=0x7f02f80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:08.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.868+0000 7f02fcaee700 1 -- 172.21.15.62:0/3158059610 shutdown_connections 2024-05-08T22:38:08.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.868+0000 7f02fcaee700 1 -- 172.21.15.62:0/3158059610 wait complete. 2024-05-08T22:38:08.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02fcaee700 1 Processor -- start 2024-05-08T22:38:08.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02fcaee700 1 -- start start 2024-05-08T22:38:08.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02fcaee700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02f8101900 0x7f02f8196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02fcaee700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8104230 0x7f02f81973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02fcaee700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02f8106b60 0x7f02f819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02fcaee700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f819e9c0 con 0x7f02f8104230 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02fcaee700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f819eb00 con 0x7f02f8106b60 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02fcaee700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f819ec90 con 0x7f02f8101900 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8104230 0x7f02f81973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8104230 0x7f02f81973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47560/0 (socket says 172.21.15.62:47560) 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f5d9b700 1 -- 172.21.15.62:0/1843303360 learned_addr learned my addr 172.21.15.62:0/1843303360 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02f8101900 0x7f02f8196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f6d9d700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02f8106b60 0x7f02f819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f659c700 1 -- 172.21.15.62:0/1843303360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02f8106b60 msgr2=0x7f02f819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f659c700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02f8106b60 0x7f02f819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f659c700 1 -- 172.21.15.62:0/1843303360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8104230 msgr2=0x7f02f81973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:08.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f659c700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8104230 0x7f02f81973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:08.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f659c700 1 -- 172.21.15.62:0/1843303360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f0009d00 con 0x7f02f8101900 2024-05-08T22:38:08.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f5d9b700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8104230 0x7f02f81973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:08.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.869+0000 7f02f659c700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02f8101900 0x7f02f8196e90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02e800e9f0 tx=0x7f02e800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:08.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.870+0000 7f02e77fe700 1 -- 172.21.15.62:0/1843303360 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800c9c0 con 0x7f02f8101900 2024-05-08T22:38:08.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.870+0000 7f02fcaee700 1 -- 172.21.15.62:0/1843303360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f819ef70 con 0x7f02f8101900 2024-05-08T22:38:08.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.870+0000 7f02e77fe700 1 -- 172.21.15.62:0/1843303360 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e8010430 con 0x7f02f8101900 2024-05-08T22:38:08.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.870+0000 7f02e77fe700 1 -- 172.21.15.62:0/1843303360 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8018680 con 0x7f02f8101900 2024-05-08T22:38:08.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.870+0000 7f02fcaee700 1 -- 172.21.15.62:0/1843303360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f819f3a0 con 0x7f02f8101900 2024-05-08T22:38:08.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.871+0000 7f02fcaee700 1 -- 172.21.15.62:0/1843303360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f80fce70 con 0x7f02f8101900 2024-05-08T22:38:08.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.872+0000 7f02e77fe700 1 -- 172.21.15.62:0/1843303360 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f02e8010860 con 0x7f02f8101900 2024-05-08T22:38:08.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.872+0000 7f02e77fe700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f02e00a1120 0x7f02e00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:08.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.873+0000 7f02f5d9b700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f02e00a1120 0x7f02e00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:08.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.873+0000 7f02e77fe700 1 -- 172.21.15.62:0/1843303360 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f02e8014070 con 0x7f02f8101900 2024-05-08T22:38:08.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.873+0000 7f02f5d9b700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f02e00a1120 0x7f02e00a35e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f02ec011550 tx=0x7f02ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:08.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:08.876+0000 7f02e77fe700 1 -- 172.21.15.62:0/1843303360 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f02e8096350 con 0x7f02f8101900 2024-05-08T22:38:09.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.091+0000 7f02fcaee700 1 -- 172.21.15.62:0/1843303360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02f810c660 con 0x7f02f8101900 2024-05-08T22:38:09.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.094+0000 7f02e77fe700 1 -- 172.21.15.62:0/1843303360 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22481 (secure 0 0 0) 0x7f02e8099970 con 0x7f02f8101900 2024-05-08T22:38:09.094 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:38:09.094 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:36:05.960694+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:38:09.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 -- 172.21.15.62:0/1843303360 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f02e00a1120 msgr2=0x7f02e00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f02e00a1120 0x7f02e00a35e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f02ec011550 tx=0x7f02ec009260 comp rx=0 tx=0).stop 2024-05-08T22:38:09.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 -- 172.21.15.62:0/1843303360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02f8101900 msgr2=0x7f02f8196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02f8101900 0x7f02f8196e90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02e800e9f0 tx=0x7f02e800ed00 comp rx=0 tx=0).stop 2024-05-08T22:38:09.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 -- 172.21.15.62:0/1843303360 shutdown_connections 2024-05-08T22:38:09.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f02e00a1120 0x7f02e00a35e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02f8106b60 0x7f02f819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02f8101900 0x7f02f8196e90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 --2- 172.21.15.62:0/1843303360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f8104230 0x7f02f81973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 -- 172.21.15.62:0/1843303360 >> 172.21.15.62:0/1843303360 conn(0x7f02f80fb2f0 msgr2=0x7f02f8102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:09.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 -- 172.21.15.62:0/1843303360 shutdown_connections 2024-05-08T22:38:09.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.096+0000 7f02fcaee700 1 -- 172.21.15.62:0/1843303360 wait complete. 2024-05-08T22:38:09.109 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:31:54.466249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:35:08.296334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:35:09.167901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:38:09.109 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.239+0000 7f7a7c17e700 1 -- 172.21.15.62:0/3729225099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a74102980 msgr2=0x7f7a741092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.239+0000 7f7a7c17e700 1 --2- 172.21.15.62:0/3729225099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a74102980 0x7f7a741092a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00b0a0 tx=0x7f7a6c00b3b0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.239+0000 7f7a7c17e700 1 -- 172.21.15.62:0/3729225099 shutdown_connections 2024-05-08T22:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.239+0000 7f7a7c17e700 1 --2- 172.21.15.62:0/3729225099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a74102980 0x7f7a741092a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.239+0000 7f7a7c17e700 1 --2- 172.21.15.62:0/3729225099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a74100e70 0x7f7a74101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.239+0000 7f7a7c17e700 1 --2- 172.21.15.62:0/3729225099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a74101fc0 0x7f7a74102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.239+0000 7f7a7c17e700 1 -- 172.21.15.62:0/3729225099 >> 172.21.15.62:0/3729225099 conn(0x7f7a740fc400 msgr2=0x7f7a740fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.239+0000 7f7a7c17e700 1 -- 172.21.15.62:0/3729225099 shutdown_connections 2024-05-08T22:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.239+0000 7f7a7c17e700 1 -- 172.21.15.62:0/3729225099 wait complete. 2024-05-08T22:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.239+0000 7f7a7c17e700 1 Processor -- start 2024-05-08T22:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.239+0000 7f7a7c17e700 1 -- start start 2024-05-08T22:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a7c17e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a74100e70 0x7f7a74196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a7c17e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a74101fc0 0x7f7a741973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a7c17e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a74102980 0x7f7a7419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a7c17e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7419e9b0 con 0x7f7a74101fc0 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a7c17e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7419eaf0 con 0x7f7a74102980 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a7c17e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7419ec80 con 0x7f7a74100e70 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a79719700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a74101fc0 0x7f7a741973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a79719700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a74101fc0 0x7f7a741973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47574/0 (socket says 172.21.15.62:47574) 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a79719700 1 -- 172.21.15.62:0/286946446 learned_addr learned my addr 172.21.15.62:0/286946446 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a79f1a700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a74100e70 0x7f7a74196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a7a71b700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a74102980 0x7f7a7419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a79719700 1 -- 172.21.15.62:0/286946446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a74100e70 msgr2=0x7f7a74196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a79719700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a74100e70 0x7f7a74196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a79719700 1 -- 172.21.15.62:0/286946446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a74102980 msgr2=0x7f7a7419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a79719700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a74102980 0x7f7a7419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a79719700 1 -- 172.21.15.62:0/286946446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c009d00 con 0x7f7a74101fc0 2024-05-08T22:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a79f1a700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a74100e70 0x7f7a74196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:09.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a7a71b700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a74102980 0x7f7a7419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:09.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a79719700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a74101fc0 0x7f7a741973c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c920 tx=0x7f7a6400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:09.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a6affd700 1 -- 172.21.15.62:0/286946446 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64007b00 con 0x7f7a74101fc0 2024-05-08T22:38:09.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a6affd700 1 -- 172.21.15.62:0/286946446 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a64004cf0 con 0x7f7a74101fc0 2024-05-08T22:38:09.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.240+0000 7f7a6affd700 1 -- 172.21.15.62:0/286946446 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64005610 con 0x7f7a74101fc0 2024-05-08T22:38:09.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.241+0000 7f7a7c17e700 1 -- 172.21.15.62:0/286946446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7419ef60 con 0x7f7a74101fc0 2024-05-08T22:38:09.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.241+0000 7f7a7c17e700 1 -- 172.21.15.62:0/286946446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7419f390 con 0x7f7a74101fc0 2024-05-08T22:38:09.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.241+0000 7f7a7c17e700 1 -- 172.21.15.62:0/286946446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a74108210 con 0x7f7a74101fc0 2024-05-08T22:38:09.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.243+0000 7f7a6affd700 1 -- 172.21.15.62:0/286946446 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7a64004750 con 0x7f7a74101fc0 2024-05-08T22:38:09.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.243+0000 7f7a6affd700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a600a1120 0x7f7a600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.244+0000 7f7a79f1a700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a600a1120 0x7f7a600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.244+0000 7f7a6affd700 1 -- 172.21.15.62:0/286946446 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a640cc820 con 0x7f7a74101fc0 2024-05-08T22:38:09.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.244+0000 7f7a79f1a700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a600a1120 0x7f7a600a35e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7a70005fd0 tx=0x7f7a70005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:09.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.248+0000 7f7a6affd700 1 -- 172.21.15.62:0/286946446 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7a64095fc0 con 0x7f7a74101fc0 2024-05-08T22:38:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.452+0000 7f7a7c17e700 1 -- 172.21.15.62:0/286946446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a7410c700 con 0x7f7a74101fc0 2024-05-08T22:38:09.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.452+0000 7f7a6affd700 1 -- 172.21.15.62:0/286946446 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a6401d020 con 0x7f7a74101fc0 2024-05-08T22:38:09.452 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-05-08T22:38:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 -- 172.21.15.62:0/286946446 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a600a1120 msgr2=0x7f7a600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a600a1120 0x7f7a600a35e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7a70005fd0 tx=0x7f7a70005f60 comp rx=0 tx=0).stop 2024-05-08T22:38:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 -- 172.21.15.62:0/286946446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a74101fc0 msgr2=0x7f7a741973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a74101fc0 0x7f7a741973c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c920 tx=0x7f7a6400cce0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 -- 172.21.15.62:0/286946446 shutdown_connections 2024-05-08T22:38:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f7a600a1120 0x7f7a600a35e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a74102980 0x7f7a7419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a74100e70 0x7f7a74196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 --2- 172.21.15.62:0/286946446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a74101fc0 0x7f7a741973c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 -- 172.21.15.62:0/286946446 >> 172.21.15.62:0/286946446 conn(0x7f7a740fc400 msgr2=0x7f7a74105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 -- 172.21.15.62:0/286946446 shutdown_connections 2024-05-08T22:38:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.455+0000 7f7a7c17e700 1 -- 172.21.15.62:0/286946446 wait complete. 2024-05-08T22:38:09.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:38:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.598+0000 7f2397480700 1 -- 172.21.15.62:0/2010163832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2390106b70 msgr2=0x7f2390108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.598+0000 7f2397480700 1 --2- 172.21.15.62:0/2010163832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2390106b70 0x7f2390108fe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f238800b0a0 tx=0x7f238800b3b0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.598+0000 7f2397480700 1 -- 172.21.15.62:0/2010163832 shutdown_connections 2024-05-08T22:38:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.598+0000 7f2397480700 1 --2- 172.21.15.62:0/2010163832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2390106b70 0x7f2390108fe0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.598+0000 7f2397480700 1 --2- 172.21.15.62:0/2010163832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2390101910 0x7f2390103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.598+0000 7f2397480700 1 --2- 172.21.15.62:0/2010163832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390104240 0x7f2390106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.598+0000 7f2397480700 1 -- 172.21.15.62:0/2010163832 >> 172.21.15.62:0/2010163832 conn(0x7f23900fb300 msgr2=0x7f23900fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.598+0000 7f2397480700 1 -- 172.21.15.62:0/2010163832 shutdown_connections 2024-05-08T22:38:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.598+0000 7f2397480700 1 -- 172.21.15.62:0/2010163832 wait complete. 2024-05-08T22:38:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2397480700 1 Processor -- start 2024-05-08T22:38:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2397480700 1 -- start start 2024-05-08T22:38:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2397480700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390101910 0x7f2390101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2397480700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2390104240 0x7f23900ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2397480700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2390106b70 0x7f23900ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2397480700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23901005a0 con 0x7f2390104240 2024-05-08T22:38:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2397480700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2390100710 con 0x7f2390101910 2024-05-08T22:38:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2397480700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23901a0f00 con 0x7f2390106b70 2024-05-08T22:38:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2394a1b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2390104240 0x7f23900ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2394a1b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2390104240 0x7f23900ff8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47598/0 (socket says 172.21.15.62:47598) 2024-05-08T22:38:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2394a1b700 1 -- 172.21.15.62:0/927609863 learned_addr learned my addr 172.21.15.62:0/927609863 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f239521c700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390101910 0x7f2390101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2395a1d700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2390106b70 0x7f23900ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2394a1b700 1 -- 172.21.15.62:0/927609863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2390106b70 msgr2=0x7f23900ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2394a1b700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2390106b70 0x7f23900ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2394a1b700 1 -- 172.21.15.62:0/927609863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390101910 msgr2=0x7f2390101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2394a1b700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390101910 0x7f2390101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2394a1b700 1 -- 172.21.15.62:0/927609863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388009d00 con 0x7f2390104240 2024-05-08T22:38:09.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.599+0000 7f2394a1b700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2390104240 0x7f23900ff8c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f238000c920 tx=0x7f238000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:09.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.600+0000 7f23867fc700 1 -- 172.21.15.62:0/927609863 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380007aa0 con 0x7f2390104240 2024-05-08T22:38:09.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.600+0000 7f2397480700 1 -- 172.21.15.62:0/927609863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23901a1190 con 0x7f2390104240 2024-05-08T22:38:09.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.600+0000 7f23867fc700 1 -- 172.21.15.62:0/927609863 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f238000ce70 con 0x7f2390104240 2024-05-08T22:38:09.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.600+0000 7f23867fc700 1 -- 172.21.15.62:0/927609863 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380005440 con 0x7f2390104240 2024-05-08T22:38:09.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.600+0000 7f2397480700 1 -- 172.21.15.62:0/927609863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23901a16a0 con 0x7f2390104240 2024-05-08T22:38:09.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.600+0000 7f2397480700 1 -- 172.21.15.62:0/927609863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23900fce80 con 0x7f2390104240 2024-05-08T22:38:09.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.602+0000 7f23867fc700 1 -- 172.21.15.62:0/927609863 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f23800055c0 con 0x7f2390104240 2024-05-08T22:38:09.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.602+0000 7f23867fc700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f237c0a1120 0x7f237c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.602+0000 7f239521c700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f237c0a1120 0x7f237c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.602+0000 7f23867fc700 1 -- 172.21.15.62:0/927609863 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23800cc820 con 0x7f2390104240 2024-05-08T22:38:09.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.603+0000 7f239521c700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f237c0a1120 0x7f237c0a35e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f238c005950 tx=0x7f238c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.606+0000 7f23867fc700 1 -- 172.21.15.62:0/927609863 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2380095fc0 con 0x7f2390104240 2024-05-08T22:38:09.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.810+0000 7f2397480700 1 -- 172.21.15.62:0/927609863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f239010c670 con 0x7f2390104240 2024-05-08T22:38:09.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.811+0000 7f23867fc700 1 -- 172.21.15.62:0/927609863 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f238001d0a0 con 0x7f2390104240 2024-05-08T22:38:09.811 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-05-08T22:38:09.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 -- 172.21.15.62:0/927609863 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f237c0a1120 msgr2=0x7f237c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f237c0a1120 0x7f237c0a35e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f238c005950 tx=0x7f238c00b410 comp rx=0 tx=0).stop 2024-05-08T22:38:09.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 -- 172.21.15.62:0/927609863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2390104240 msgr2=0x7f23900ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2390104240 0x7f23900ff8c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f238000c920 tx=0x7f238000cc30 comp rx=0 tx=0).stop 2024-05-08T22:38:09.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 -- 172.21.15.62:0/927609863 shutdown_connections 2024-05-08T22:38:09.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2390106b70 0x7f23900ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f237c0a1120 0x7f237c0a35e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2390101910 0x7f2390101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 --2- 172.21.15.62:0/927609863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2390104240 0x7f23900ff8c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 -- 172.21.15.62:0/927609863 >> 172.21.15.62:0/927609863 conn(0x7f23900fb300 msgr2=0x7f23901021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:09.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 -- 172.21.15.62:0/927609863 shutdown_connections 2024-05-08T22:38:09.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.813+0000 7f2397480700 1 -- 172.21.15.62:0/927609863 wait complete. 2024-05-08T22:38:09.824 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:09 smithi062 ceph-mon[42571]: pgmap v313: 97 pgs: 97 active+clean; 1.6 GiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 1.31k op/s 2024-05-08T22:38:09.824 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:09 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1843303360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:38:09.824 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:09 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/286946446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:38:09.825 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:38:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:09 smithi079 ceph-mon[46738]: pgmap v313: 97 pgs: 97 active+clean; 1.6 GiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 1.31k op/s 2024-05-08T22:38:09.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:09 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1843303360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:38:09.889 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:09 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/286946446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:38:09.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.955+0000 7fc88e765700 1 -- 172.21.15.62:0/316411007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc888101930 msgr2=0x7fc8881081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.955+0000 7fc88e765700 1 --2- 172.21.15.62:0/316411007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc888101930 0x7fc8881081e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc88000b0d0 tx=0x7fc88000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.955+0000 7fc88e765700 1 -- 172.21.15.62:0/316411007 shutdown_connections 2024-05-08T22:38:09.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.955+0000 7fc88e765700 1 --2- 172.21.15.62:0/316411007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc888101930 0x7fc8881081e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.955+0000 7fc88e765700 1 --2- 172.21.15.62:0/316411007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8880ffda0 0x7fc8881001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.955+0000 7fc88e765700 1 --2- 172.21.15.62:0/316411007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc888100f90 0x7fc8881013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.955+0000 7fc88e765700 1 -- 172.21.15.62:0/316411007 >> 172.21.15.62:0/316411007 conn(0x7fc8880fb2d0 msgr2=0x7fc8880fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:09.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.955+0000 7fc88e765700 1 -- 172.21.15.62:0/316411007 shutdown_connections 2024-05-08T22:38:09.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.955+0000 7fc88e765700 1 -- 172.21.15.62:0/316411007 wait complete. 2024-05-08T22:38:09.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.956+0000 7fc88e765700 1 Processor -- start 2024-05-08T22:38:09.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.956+0000 7fc88e765700 1 -- start start 2024-05-08T22:38:09.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.956+0000 7fc88e765700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8880ffda0 0x7fc888103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.956+0000 7fc88e765700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc888100f90 0x7fc888102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.956+0000 7fc88e765700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc888101930 0x7fc8881028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.956+0000 7fc88e765700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc888102f10 con 0x7fc888100f90 2024-05-08T22:38:09.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.956+0000 7fc88e765700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc888103080 con 0x7fc8880ffda0 2024-05-08T22:38:09.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.956+0000 7fc88e765700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc888103240 con 0x7fc888101930 2024-05-08T22:38:09.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.956+0000 7fc8877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc888100f90 0x7fc888102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.956+0000 7fc8877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc888100f90 0x7fc888102370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47620/0 (socket says 172.21.15.62:47620) 2024-05-08T22:38:09.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.956+0000 7fc8877fe700 1 -- 172.21.15.62:0/289232968 learned_addr learned my addr 172.21.15.62:0/289232968 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:09.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc88cd02700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc888101930 0x7fc8881028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc887fff700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8880ffda0 0x7fc888103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc8877fe700 1 -- 172.21.15.62:0/289232968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc888101930 msgr2=0x7fc8881028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc8877fe700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc888101930 0x7fc8881028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc8877fe700 1 -- 172.21.15.62:0/289232968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8880ffda0 msgr2=0x7fc888103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc8877fe700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8880ffda0 0x7fc888103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc8877fe700 1 -- 172.21.15.62:0/289232968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc880009d00 con 0x7fc888100f90 2024-05-08T22:38:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc887fff700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8880ffda0 0x7fc888103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc8877fe700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc888100f90 0x7fc888102370 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00d950 tx=0x7fc87c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc8857fa700 1 -- 172.21.15.62:0/289232968 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c009940 con 0x7fc888100f90 2024-05-08T22:38:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc88e765700 1 -- 172.21.15.62:0/289232968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8881a1250 con 0x7fc888100f90 2024-05-08T22:38:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc8857fa700 1 -- 172.21.15.62:0/289232968 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc87c010460 con 0x7fc888100f90 2024-05-08T22:38:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc8857fa700 1 -- 172.21.15.62:0/289232968 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00f620 con 0x7fc888100f90 2024-05-08T22:38:09.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.957+0000 7fc88e765700 1 -- 172.21.15.62:0/289232968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8881a1760 con 0x7fc888100f90 2024-05-08T22:38:09.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.958+0000 7fc88e765700 1 -- 172.21.15.62:0/289232968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc888067480 con 0x7fc888100f90 2024-05-08T22:38:09.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.959+0000 7fc8857fa700 1 -- 172.21.15.62:0/289232968 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc87c0108d0 con 0x7fc888100f90 2024-05-08T22:38:09.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.960+0000 7fc8857fa700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc8700a1120 0x7fc8700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:09.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.960+0000 7fc8857fa700 1 -- 172.21.15.62:0/289232968 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc87c0cc6d0 con 0x7fc888100f90 2024-05-08T22:38:09.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.960+0000 7fc887fff700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc8700a1120 0x7fc8700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:09.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.961+0000 7fc887fff700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc8700a1120 0x7fc8700a35e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc888100df0 tx=0x7fc87800b670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:09.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:09.964+0000 7fc8857fa700 1 -- 172.21.15.62:0/289232968 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc87c095e70 con 0x7fc888100f90 2024-05-08T22:38:10.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:09 smithi066 ceph-mon[46830]: pgmap v313: 97 pgs: 97 active+clean; 1.6 GiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 1.31k op/s 2024-05-08T22:38:10.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:09 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1843303360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:38:10.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:09 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/286946446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:38:10.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.175+0000 7fc88e765700 1 -- 172.21.15.62:0/289232968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8881a1b50 con 0x7fc888100f90 2024-05-08T22:38:10.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.176+0000 7fc8857fa700 1 -- 172.21.15.62:0/289232968 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc87c099490 con 0x7fc888100f90 2024-05-08T22:38:10.176 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-05-08T22:38:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 -- 172.21.15.62:0/289232968 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc8700a1120 msgr2=0x7fc8700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc8700a1120 0x7fc8700a35e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc888100df0 tx=0x7fc87800b670 comp rx=0 tx=0).stop 2024-05-08T22:38:10.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 -- 172.21.15.62:0/289232968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc888100f90 msgr2=0x7fc888102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:10.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc888100f90 0x7fc888102370 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00d950 tx=0x7fc87c00dc60 comp rx=0 tx=0).stop 2024-05-08T22:38:10.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 -- 172.21.15.62:0/289232968 shutdown_connections 2024-05-08T22:38:10.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc888101930 0x7fc8881028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fc8700a1120 0x7fc8700a35e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8880ffda0 0x7fc888103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 --2- 172.21.15.62:0/289232968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc888100f90 0x7fc888102370 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 -- 172.21.15.62:0/289232968 >> 172.21.15.62:0/289232968 conn(0x7fc8880fb2d0 msgr2=0x7fc8880fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:10.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 -- 172.21.15.62:0/289232968 shutdown_connections 2024-05-08T22:38:10.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.178+0000 7fc88e765700 1 -- 172.21.15.62:0/289232968 wait complete. 2024-05-08T22:38:10.191 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-08T22:38:10.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.317+0000 7fba2597b700 1 -- 172.21.15.62:0/2087477314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba20101960 msgr2=0x7fba20103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:10.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.317+0000 7fba2597b700 1 --2- 172.21.15.62:0/2087477314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba20101960 0x7fba20103d50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fba10009990 tx=0x7fba10009ca0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.317+0000 7fba2597b700 1 -- 172.21.15.62:0/2087477314 shutdown_connections 2024-05-08T22:38:10.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.317+0000 7fba2597b700 1 --2- 172.21.15.62:0/2087477314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba20106bc0 0x7fba20108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.317+0000 7fba2597b700 1 --2- 172.21.15.62:0/2087477314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba20101960 0x7fba20103d50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.317+0000 7fba2597b700 1 --2- 172.21.15.62:0/2087477314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba20104290 0x7fba20106680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.317+0000 7fba2597b700 1 -- 172.21.15.62:0/2087477314 >> 172.21.15.62:0/2087477314 conn(0x7fba200fb310 msgr2=0x7fba200fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:10.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.317+0000 7fba2597b700 1 -- 172.21.15.62:0/2087477314 shutdown_connections 2024-05-08T22:38:10.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.318+0000 7fba2597b700 1 -- 172.21.15.62:0/2087477314 wait complete. 2024-05-08T22:38:10.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.318+0000 7fba2597b700 1 Processor -- start 2024-05-08T22:38:10.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.318+0000 7fba2597b700 1 -- start start 2024-05-08T22:38:10.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.318+0000 7fba2597b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba20101960 0x7fba20196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:10.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.318+0000 7fba2597b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba20104290 0x7fba201972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:10.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.318+0000 7fba2597b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba20106bc0 0x7fba2019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:10.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.318+0000 7fba2597b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2019e8e0 con 0x7fba20104290 2024-05-08T22:38:10.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.318+0000 7fba2597b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2019ea20 con 0x7fba20101960 2024-05-08T22:38:10.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.318+0000 7fba2597b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2019ebb0 con 0x7fba20106bc0 2024-05-08T22:38:10.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba20104290 0x7fba201972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:10.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba20104290 0x7fba201972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47634/0 (socket says 172.21.15.62:47634) 2024-05-08T22:38:10.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1e7fc700 1 -- 172.21.15.62:0/1951164789 learned_addr learned my addr 172.21.15.62:0/1951164789 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:10.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1f7fe700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba20106bc0 0x7fba2019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:10.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1effd700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba20101960 0x7fba20196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:10.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1e7fc700 1 -- 172.21.15.62:0/1951164789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba20106bc0 msgr2=0x7fba2019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:10.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1e7fc700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba20106bc0 0x7fba2019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1e7fc700 1 -- 172.21.15.62:0/1951164789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba20101960 msgr2=0x7fba20196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:10.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1e7fc700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba20101960 0x7fba20196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1e7fc700 1 -- 172.21.15.62:0/1951164789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba100095f0 con 0x7fba20104290 2024-05-08T22:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1f7fe700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba20106bc0 0x7fba2019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba1e7fc700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba20104290 0x7fba201972f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fba1400d950 tx=0x7fba1400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba24979700 1 -- 172.21.15.62:0/1951164789 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14009b40 con 0x7fba20104290 2024-05-08T22:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba24979700 1 -- 172.21.15.62:0/1951164789 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba14009ca0 con 0x7fba20104290 2024-05-08T22:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba2597b700 1 -- 172.21.15.62:0/1951164789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2019ee90 con 0x7fba20104290 2024-05-08T22:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba24979700 1 -- 172.21.15.62:0/1951164789 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400f620 con 0x7fba20104290 2024-05-08T22:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.319+0000 7fba2597b700 1 -- 172.21.15.62:0/1951164789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2019f310 con 0x7fba20104290 2024-05-08T22:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.320+0000 7fba2597b700 1 -- 172.21.15.62:0/1951164789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba200fce90 con 0x7fba20104290 2024-05-08T22:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.322+0000 7fba24979700 1 -- 172.21.15.62:0/1951164789 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fba14010460 con 0x7fba20104290 2024-05-08T22:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.322+0000 7fba24979700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba080a1120 0x7fba080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.322+0000 7fba24979700 1 -- 172.21.15.62:0/1951164789 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fba140ccc50 con 0x7fba20104290 2024-05-08T22:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.322+0000 7fba1effd700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba080a1120 0x7fba080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.323+0000 7fba1effd700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba080a1120 0x7fba080a35e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba1000b2e0 tx=0x7fba10005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.326+0000 7fba24979700 1 -- 172.21.15.62:0/1951164789 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fba140963f0 con 0x7fba20104290 2024-05-08T22:38:10.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.600+0000 7fba2597b700 1 -- 172.21.15.62:0/1951164789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fba2010c660 con 0x7fba20104290 2024-05-08T22:38:10.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.607+0000 7fba24979700 1 -- 172.21.15.62:0/1951164789 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v17) v1 ==== 81+0+733410 (secure 0 0 0) 0x7fba14020090 con 0x7fba20104290 2024-05-08T22:38:10.617 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:38:10.618 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:38:10.618 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 17, 2024-05-08T22:38:10.618 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.618 INFO:teuthology.orchestra.run.smithi062.stdout: "active_gid": 14150, 2024-05-08T22:38:10.618 INFO:teuthology.orchestra.run.smithi062.stdout: "active_name": "x", 2024-05-08T22:38:10.618 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addrs": { 2024-05-08T22:38:10.618 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-05-08T22:38:10.618 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.618 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-05-08T22:38:10.618 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6800", 2024-05-08T22:38:10.618 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 4071146880 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6801", 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 4071146880 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addr": "172.21.15.62:6801/4071146880", 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: "active_change": "2024-05-08T22:30:03.951233+0000", 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: "active_mgr_features": 4540138322906710015, 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: "available": true, 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: "standbys": [ 2024-05-08T22:38:10.619 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 24109, 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "z", 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540138322906710015, 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.621 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.622 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.623 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.624 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.625 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.626 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-05-08T22:38:10.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-05-08T22:38:10.628 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-05-08T22:38:10.629 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.630 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-05-08T22:38:10.631 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.632 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.633 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.634 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-05-08T22:38:10.634 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-05-08T22:38:10.634 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.634 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.634 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.634 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.634 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.634 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.634 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.634 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.635 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.636 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-05-08T22:38:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-05-08T22:38:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-05-08T22:38:10.640 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-05-08T22:38:10.641 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-05-08T22:38:10.642 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.643 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.645 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.646 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.648 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap" 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.650 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.651 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.652 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.653 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:10.655 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.656 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.657 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.659 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-05-08T22:38:10.660 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.661 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.662 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.663 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.664 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.665 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.666 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T22:38:10.667 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T22:38:10.668 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T22:38:10.669 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.670 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.672 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.673 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-05-08T22:38:10.674 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.675 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.676 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.677 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.678 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.679 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.680 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.681 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.682 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.683 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.684 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:10.685 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:10.686 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.687 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.688 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.689 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-05-08T22:38:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.691 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.692 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.693 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.694 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T22:38:10.695 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T22:38:10.696 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.697 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.698 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.699 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-05-08T22:38:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.701 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.702 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.703 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.704 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.705 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.706 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.707 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-05-08T22:38:10.708 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.709 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.710 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.711 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.712 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.713 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.714 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.715 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.716 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.717 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.718 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.720 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.721 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.722 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.724 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.725 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.726 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.727 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.728 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.729 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.730 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.731 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.733 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.734 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.734 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.734 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T22:38:10.734 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T22:38:10.734 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.734 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.734 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.734 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.734 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.734 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.735 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.736 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.737 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.740 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.741 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.742 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.743 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.744 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.745 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.746 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.747 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.749 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.750 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.751 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.752 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.753 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.754 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.755 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.756 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.757 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.758 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.759 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.760 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.761 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.762 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.763 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.764 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.765 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.766 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.767 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.768 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.769 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.770 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.771 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.772 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.773 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.774 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.775 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.776 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-05-08T22:38:10.777 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.778 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.779 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.780 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.780 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.780 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-05-08T22:38:10.780 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-05-08T22:38:10.780 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.780 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.781 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.781 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.782 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.783 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-05-08T22:38:10.785 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.786 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.787 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.787 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.787 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.787 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.787 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.787 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-05-08T22:38:10.788 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.790 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.791 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.792 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.793 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.794 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.795 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.796 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.800 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.802 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.803 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-05-08T22:38:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-05-08T22:38:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-05-08T22:38:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T22:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T22:38:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T22:38:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T22:38:10.837 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.838 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.839 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.840 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.841 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.842 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-05-08T22:38:10.843 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.844 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.845 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.846 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.847 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.848 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-05-08T22:38:10.849 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-05-08T22:38:10.850 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-05-08T22:38:10.851 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-05-08T22:38:10.852 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-05-08T22:38:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.855 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.856 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.857 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.858 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.859 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.860 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.861 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.862 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.863 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.864 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.865 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.866 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.867 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.868 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.869 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.870 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.871 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.873 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.874 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.876 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.877 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.880 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.881 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.882 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.883 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.884 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.885 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.886 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.887 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.888 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.890 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.891 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.892 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.893 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.894 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.895 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.896 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.897 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-05-08T22:38:10.898 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.899 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.900 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.901 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.903 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-05-08T22:38:10.904 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:10.905 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.906 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.907 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.908 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.909 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.910 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.911 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.912 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.914 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.915 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.916 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T22:38:10.917 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T22:38:10.918 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.919 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-05-08T22:38:10.920 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-05-08T22:38:10.921 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.922 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.924 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.925 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.926 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-05-08T22:38:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T22:38:10.928 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-05-08T22:38:10.929 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.930 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.931 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.932 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.933 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.934 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.935 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.936 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.936 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.936 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.936 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.936 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.936 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.936 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.936 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.936 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.937 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-05-08T22:38:10.938 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.939 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.940 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.941 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.942 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.943 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.944 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.945 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.946 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.947 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.948 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.949 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.950 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.951 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-05-08T22:38:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.953 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.955 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.956 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.957 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.958 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.959 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.961 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.962 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.964 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.966 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.967 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.968 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.969 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.970 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.971 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.972 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.973 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.974 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.975 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.977 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.978 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.979 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.980 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.981 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.982 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.983 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.984 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.985 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.986 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.987 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.987 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.987 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.987 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.987 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.987 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.987 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.987 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.987 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.987 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.987 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.989 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.989 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-05-08T22:38:10.990 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.991 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.992 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.993 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.994 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:10.995 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:10.996 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.997 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.998 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:10.999 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.000 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.001 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.002 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.003 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.004 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.005 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.006 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-05-08T22:38:11.007 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.008 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.012 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.013 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.014 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.015 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.016 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.017 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.018 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.019 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.020 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.021 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.022 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.023 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.024 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.025 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.027 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.029 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.030 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.031 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.032 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:11.033 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-05-08T22:38:11.034 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.036 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.037 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.038 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.039 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.040 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.041 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-05-08T22:38:11.042 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-05-08T22:38:11.043 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 24119, 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "y", 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540138322906710015, 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.045 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.046 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.048 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.049 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.050 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.051 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-05-08T22:38:11.052 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.053 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.054 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.056 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.058 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-05-08T22:38:11.059 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.060 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.062 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.063 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.064 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.065 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.067 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.068 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.069 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.070 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-05-08T22:38:11.072 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-05-08T22:38:11.073 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap" 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.074 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.075 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T22:38:11.076 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T22:38:11.077 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T22:38:11.078 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-05-08T22:38:11.079 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-05-08T22:38:11.080 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-05-08T22:38:11.081 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-05-08T22:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-05-08T22:38:11.083 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.084 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-05-08T22:38:11.085 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-05-08T22:38:11.086 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-05-08T22:38:11.087 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-05-08T22:38:11.088 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-05-08T22:38:11.089 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-05-08T22:38:11.091 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.092 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.093 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.094 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.095 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.097 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.098 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.099 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-05-08T22:38:11.100 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-05-08T22:38:11.102 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-05-08T22:38:11.103 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.105 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.106 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.107 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.108 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.109 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.110 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.111 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.112 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.113 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.115 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.116 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.117 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.118 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.119 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.121 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.122 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.123 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.124 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T22:38:11.125 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.126 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T22:38:11.127 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.128 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.129 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.130 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.131 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.132 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.133 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.134 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.135 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.136 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.138 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.139 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.140 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.141 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.142 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.143 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.144 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.145 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.146 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.147 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.148 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.149 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.150 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.151 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.152 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.153 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.154 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.155 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.156 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.157 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T22:38:11.158 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T22:38:11.159 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T22:38:11.160 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T22:38:11.161 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T22:38:11.162 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-05-08T22:38:11.163 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T22:38:11.164 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T22:38:11.165 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T22:38:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T22:38:11.167 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T22:38:11.168 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T22:38:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T22:38:11.170 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.171 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.172 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.173 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.174 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.175 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.176 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.177 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.178 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T22:38:11.179 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.180 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.181 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.183 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.185 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.186 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.187 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.188 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.189 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.190 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.191 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.191 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.191 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.191 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.191 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.191 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.194 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.195 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.196 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.197 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.198 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.199 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.200 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.202 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.203 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-05-08T22:38:11.204 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-05-08T22:38:11.205 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-05-08T22:38:11.206 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.207 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-05-08T22:38:11.208 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.209 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.210 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.211 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.212 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.213 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.214 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.215 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.216 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.217 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.218 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.219 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.220 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.221 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.222 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.223 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.224 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T22:38:11.225 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.226 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.227 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.229 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.230 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.231 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.233 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-05-08T22:38:11.234 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.235 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.236 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.237 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.238 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.239 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.240 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.241 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.242 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.243 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.244 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.245 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.246 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.247 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.248 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.249 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.250 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.251 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.252 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.253 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.254 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.255 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.256 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.257 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.258 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.259 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.260 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.261 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.262 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.264 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.265 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.267 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.268 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.269 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.270 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.271 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.272 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.273 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.275 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.277 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.278 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.279 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.280 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.281 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.282 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.283 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.283 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.283 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.283 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.283 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.283 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.283 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.283 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.283 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.283 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.283 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.284 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.285 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.286 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.287 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.288 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.289 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.290 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.291 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.292 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.293 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.295 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-05-08T22:38:11.296 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.297 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.298 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.299 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.300 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.301 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-05-08T22:38:11.302 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.303 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.304 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.305 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.306 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.307 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.308 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.309 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.310 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.311 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.312 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.313 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.315 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.316 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.317 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.318 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.319 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.320 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T22:38:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.322 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.323 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.324 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.325 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.326 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.327 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.328 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-05-08T22:38:11.329 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.330 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.331 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.332 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.334 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.335 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.336 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.337 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.338 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.339 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-05-08T22:38:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-05-08T22:38:11.341 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-05-08T22:38:11.342 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-05-08T22:38:11.343 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-05-08T22:38:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.345 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.346 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.347 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.348 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.349 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.350 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.351 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.352 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.353 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.354 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-05-08T22:38:11.355 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-05-08T22:38:11.356 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.357 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.358 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.359 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.360 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.361 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.362 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.364 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.365 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.366 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.367 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.369 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.370 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.371 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.372 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.375 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.377 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.378 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.379 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.380 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-05-08T22:38:11.381 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-05-08T22:38:11.382 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-05-08T22:38:11.383 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-05-08T22:38:11.384 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-05-08T22:38:11.385 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-05-08T22:38:11.386 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-05-08T22:38:11.387 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.388 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-05-08T22:38:11.389 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-05-08T22:38:11.390 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-05-08T22:38:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-05-08T22:38:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.393 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.394 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.395 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.396 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.397 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.398 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.399 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.400 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.401 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.402 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.403 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.405 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.406 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.407 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.409 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.410 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.412 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.413 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.414 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.415 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.416 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.417 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.418 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.419 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.420 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-05-08T22:38:11.421 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-05-08T22:38:11.422 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-05-08T22:38:11.423 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-05-08T22:38:11.424 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-05-08T22:38:11.425 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-05-08T22:38:11.426 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-05-08T22:38:11.427 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.428 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.429 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.431 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.432 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.435 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.436 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.438 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.439 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.440 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.441 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.442 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.443 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.444 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.446 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.447 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.449 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.450 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.451 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:11.452 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.453 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.454 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.455 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.456 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.458 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.459 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.460 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.461 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.462 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.463 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.464 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.465 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: "modules": [ 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard", 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: "iostat", 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: "nfs", 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: "restful" 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.466 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.467 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.468 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.469 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.471 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.472 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-05-08T22:38:11.473 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-05-08T22:38:11.474 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-05-08T22:38:11.475 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-05-08T22:38:11.476 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.477 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.478 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.479 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.480 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.481 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.483 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.484 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.485 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.486 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.487 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.488 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.489 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.490 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.491 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.492 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap" 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-05-08T22:38:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.500 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.501 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.502 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.503 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.504 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.505 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.506 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.507 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.508 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.509 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.510 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.511 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.512 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.513 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.514 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.515 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.516 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.517 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.518 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.519 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.520 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.521 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.523 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.524 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.525 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.526 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.527 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.530 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.531 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-05-08T22:38:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T22:38:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.536 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.537 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.539 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.540 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T22:38:11.541 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.543 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T22:38:11.544 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-05-08T22:38:11.545 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-05-08T22:38:11.546 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.547 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.548 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.549 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.550 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.551 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.552 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.553 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.554 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.555 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.556 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T22:38:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.559 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.560 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.561 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.562 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.563 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.564 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.565 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.566 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.567 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-05-08T22:38:11.568 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.569 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.570 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.571 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.574 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T22:38:11.576 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T22:38:11.577 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T22:38:11.578 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T22:38:11.579 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T22:38:11.580 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.581 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T22:38:11.583 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.584 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.585 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.586 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.588 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.595 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.596 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.597 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.598 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.599 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T22:38:11.600 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.601 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-05-08T22:38:11.602 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.603 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.604 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.605 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.605 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.605 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.605 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.605 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.605 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.605 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.605 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.605 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T22:38:11.605 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.606 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T22:38:11.607 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.609 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.610 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.611 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-05-08T22:38:11.612 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-05-08T22:38:11.613 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-05-08T22:38:11.614 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-05-08T22:38:11.615 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-05-08T22:38:11.616 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.617 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.618 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.619 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.620 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.621 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.622 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.623 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.625 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.626 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.627 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.628 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-05-08T22:38:11.629 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-05-08T22:38:11.630 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.631 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.633 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.636 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.636 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.636 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.636 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.636 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.636 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.636 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.636 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.637 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.638 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.638 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.638 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.638 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.638 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.638 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.638 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.638 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.638 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.638 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.638 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T22:38:11.639 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.640 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-05-08T22:38:11.641 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.642 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-05-08T22:38:11.643 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.644 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.644 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.644 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.644 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.644 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.644 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.644 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T22:38:11.644 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.644 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.644 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.644 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.645 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-05-08T22:38:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-05-08T22:38:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-05-08T22:38:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.647 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.648 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.649 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.649 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.649 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.649 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.649 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.649 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.649 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.649 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.649 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.649 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.650 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.650 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.650 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.650 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.650 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.650 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.650 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.650 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.650 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.650 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.652 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-05-08T22:38:11.653 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-05-08T22:38:11.654 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.654 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.654 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.654 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-05-08T22:38:11.654 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.654 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.654 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.654 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.654 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.654 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.655 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.655 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-05-08T22:38:11.655 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-05-08T22:38:11.655 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.655 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.655 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.655 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-05-08T22:38:11.655 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.655 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.655 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.656 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.656 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.656 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.656 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.656 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.656 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:11.656 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:11.656 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.656 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.656 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.656 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-05-08T22:38:11.657 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.657 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.657 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.657 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.657 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.657 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.657 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.657 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.657 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.657 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.658 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-05-08T22:38:11.658 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-05-08T22:38:11.658 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-05-08T22:38:11.658 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.658 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-05-08T22:38:11.658 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-05-08T22:38:11.658 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.658 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.658 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.658 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-05-08T22:38:11.658 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.659 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.659 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.659 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T22:38:11.659 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.659 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.659 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.659 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.659 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-05-08T22:38:11.659 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-05-08T22:38:11.659 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.659 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.660 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.660 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-05-08T22:38:11.660 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.660 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.660 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.660 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T22:38:11.660 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.660 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.660 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.660 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.660 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.662 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.662 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.662 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.662 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.662 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.662 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.662 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-05-08T22:38:11.662 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-05-08T22:38:11.662 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.662 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.662 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T22:38:11.663 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.663 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.663 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.663 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.663 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.663 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.663 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.663 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.663 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.663 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.663 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.664 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.665 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.665 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.665 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.665 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.665 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.665 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.665 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.665 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.665 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.665 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.665 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.666 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.666 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.666 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.666 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.666 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.666 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.666 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.666 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.666 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.666 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.666 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.667 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.667 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.667 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.667 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.667 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.667 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.667 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.667 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.667 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.667 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.667 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.668 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.669 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.669 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.669 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.669 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-05-08T22:38:11.669 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-05-08T22:38:11.669 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.669 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.669 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.669 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.670 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-05-08T22:38:11.671 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-05-08T22:38:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.673 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.674 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-05-08T22:38:11.674 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-05-08T22:38:11.674 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.674 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.674 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.674 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.674 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.674 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.674 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.674 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-05-08T22:38:11.674 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.675 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.676 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.677 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.678 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.679 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.679 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.679 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.679 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.679 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.679 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.679 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.679 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.679 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.679 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.679 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.680 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.680 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.680 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.680 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.680 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.680 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.680 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.680 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.680 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.680 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.680 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.681 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.682 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.682 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.682 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.682 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.682 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.682 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.682 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.682 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.682 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.682 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.682 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-05-08T22:38:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.683 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.684 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.685 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.686 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.687 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.689 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.690 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.690 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-05-08T22:38:11.690 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-05-08T22:38:11.690 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.690 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.690 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.690 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-05-08T22:38:11.690 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.690 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.690 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.690 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T22:38:11.691 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.691 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.691 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.691 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-05-08T22:38:11.691 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-05-08T22:38:11.691 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.691 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.691 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.691 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.691 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-05-08T22:38:11.692 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.692 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.692 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T22:38:11.692 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.692 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.692 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.692 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.692 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.692 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.692 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.692 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.693 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.694 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.696 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.697 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.697 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.697 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.697 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.697 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.697 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.697 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.697 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.697 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.697 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.697 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-05-08T22:38:11.698 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.699 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.700 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.702 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.703 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.704 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.705 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-05-08T22:38:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.707 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-05-08T22:38:11.708 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.709 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.710 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.710 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.710 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.710 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-05-08T22:38:11.710 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.710 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.710 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.710 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-05-08T22:38:11.710 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-05-08T22:38:11.710 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.711 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.711 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.711 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-05-08T22:38:11.711 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.711 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.711 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.711 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T22:38:11.711 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T22:38:11.711 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.711 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.711 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.712 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.712 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-05-08T22:38:11.712 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.712 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.712 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.712 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.712 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.712 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.713 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.713 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.713 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.713 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.713 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.713 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.713 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.713 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.713 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.713 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.713 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.714 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.715 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.715 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.715 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.715 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.715 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.715 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.715 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.715 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.715 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.715 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.717 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.718 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.718 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.718 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.718 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.718 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.718 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.718 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.718 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.718 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.718 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.718 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.719 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.719 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.719 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-05-08T22:38:11.719 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.719 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.719 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.719 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.719 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.719 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.719 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.719 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.720 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.720 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.720 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.720 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.720 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.720 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.720 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.720 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.720 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.720 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.720 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.721 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.722 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.723 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.723 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.723 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.723 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.723 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.723 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.723 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.723 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.723 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.723 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.723 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.724 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.724 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.724 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.724 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.724 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.724 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.724 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.724 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.724 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.724 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.724 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-05-08T22:38:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.726 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.726 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.726 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.726 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.726 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.726 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.726 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.726 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.726 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.726 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.726 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.727 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.730 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.730 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.730 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.730 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.730 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.730 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.730 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.730 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.730 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.730 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.730 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.731 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.732 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.733 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.733 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.733 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.733 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.733 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T22:38:11.733 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.733 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.733 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.733 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.733 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.733 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.734 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.734 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.734 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.734 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.734 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.734 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.734 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.734 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.734 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.734 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.734 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.737 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.738 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.739 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.739 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.739 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.739 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.739 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.739 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.739 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.739 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.739 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.739 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.739 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-05-08T22:38:11.740 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.741 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.742 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.742 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.742 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.742 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.742 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.742 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.742 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.742 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.742 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.742 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.743 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.743 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.743 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.743 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.743 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.743 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.743 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.743 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.743 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.743 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.743 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.744 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.745 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.746 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.747 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.748 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.749 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.750 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.751 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.753 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-05-08T22:38:11.754 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.755 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.756 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.757 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.758 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.758 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.758 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.758 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.758 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.758 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.758 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.758 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.758 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.758 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.758 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.759 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.760 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.761 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-05-08T22:38:11.762 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.763 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.764 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.765 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.766 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.767 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.768 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.769 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.770 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.770 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.770 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.770 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.770 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.770 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.770 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.770 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.770 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.770 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.771 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.772 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-05-08T22:38:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-05-08T22:38:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.775 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T22:38:11.776 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.777 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.778 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.778 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.778 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.778 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.778 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.778 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.778 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.778 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.778 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.778 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.778 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.779 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.780 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.781 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.783 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.784 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.785 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-05-08T22:38:11.786 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.787 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-05-08T22:38:11.789 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.790 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.790 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.790 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-05-08T22:38:11.790 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.790 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.790 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.790 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-05-08T22:38:11.790 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-05-08T22:38:11.790 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.790 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.790 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.791 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.792 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.792 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.792 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.792 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.792 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.792 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.792 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-05-08T22:38:11.792 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.792 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.792 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.793 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.794 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.795 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.796 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.797 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-05-08T22:38:11.798 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-05-08T22:38:11.799 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.801 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.802 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-05-08T22:38:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.805 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.806 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.807 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.808 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-05-08T22:38:11.810 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.811 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.812 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.813 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.813 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.813 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.813 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.813 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-05-08T22:38:11.813 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.813 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.813 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.813 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.813 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.813 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.814 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.815 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.816 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.816 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.816 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.816 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.816 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.816 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.816 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.816 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.816 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.816 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.816 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.817 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.818 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.818 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.818 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.818 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.818 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.818 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.818 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.818 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.818 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.818 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.818 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.819 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.819 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.819 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.819 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.819 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.819 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.819 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.819 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.819 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.819 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.819 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.820 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.821 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.821 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.821 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.821 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.821 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.821 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.821 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.821 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.821 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.821 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.821 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.822 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.822 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.822 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.822 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.822 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.822 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.822 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.822 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.822 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.822 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.822 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.823 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.824 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.825 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.825 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.825 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.825 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.825 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.825 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.825 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.825 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.825 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.825 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.825 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-05-08T22:38:11.826 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-05-08T22:38:11.827 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.828 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.828 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.828 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.828 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.828 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.828 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.828 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.828 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.828 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.828 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.828 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.829 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.831 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.831 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.831 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.831 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.831 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.831 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.831 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.831 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.831 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.831 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.831 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.832 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.833 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.833 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.833 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.833 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.833 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.833 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.833 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.833 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.833 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.833 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.834 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.835 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.835 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.835 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.835 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.835 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.835 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.835 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-05-08T22:38:11.835 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-05-08T22:38:11.835 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.835 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-05-08T22:38:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.837 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.838 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.839 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.839 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-05-08T22:38:11.839 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-05-08T22:38:11.839 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-05-08T22:38:11.839 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.839 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.839 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.839 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.839 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.839 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.839 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.840 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.841 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.842 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.843 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.843 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.843 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.843 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.843 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.843 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.843 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.843 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.843 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.843 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.843 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.844 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.845 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-05-08T22:38:11.845 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-05-08T22:38:11.845 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.845 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.845 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.845 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.845 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.845 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.845 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.845 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.846 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.846 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.846 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.846 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.846 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-05-08T22:38:11.846 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-05-08T22:38:11.846 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.846 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.846 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.846 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.846 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.847 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.848 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.849 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.850 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.850 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T22:38:11.850 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.850 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.850 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.850 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.850 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-05-08T22:38:11.850 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-05-08T22:38:11.850 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.850 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.850 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.851 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.852 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.853 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.854 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.854 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.854 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.854 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.854 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.854 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.854 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.854 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.854 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.854 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.854 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.855 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.855 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.855 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.855 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.855 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.855 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.855 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.855 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.855 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.855 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.855 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.856 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.857 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.858 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.859 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.860 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.861 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.861 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.861 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.861 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.861 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.861 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.861 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.861 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.861 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.861 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.861 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.863 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.863 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.863 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.863 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.863 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.863 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.863 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.863 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.863 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.863 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.865 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.865 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.865 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.865 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.865 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.865 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.865 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.865 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.865 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.865 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.865 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.866 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.866 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.866 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.866 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.866 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.866 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.866 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.866 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.866 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.866 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.867 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.867 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.867 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.867 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.867 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.867 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.867 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.867 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.867 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.867 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.868 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.869 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.871 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.872 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.873 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.874 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.875 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.875 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.875 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.875 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.875 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.875 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.875 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.875 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.875 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.875 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.875 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.876 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.877 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.879 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.880 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-05-08T22:38:11.880 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.880 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.880 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.880 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-05-08T22:38:11.880 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-05-08T22:38:11.880 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.880 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.880 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.880 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.881 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.881 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.881 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.881 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T22:38:11.881 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.881 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.881 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.881 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.881 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-05-08T22:38:11.881 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-05-08T22:38:11.881 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-05-08T22:38:11.883 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-05-08T22:38:11.883 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.883 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.883 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.883 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.883 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.883 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.883 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.883 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T22:38:11.883 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.883 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.884 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.884 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-05-08T22:38:11.884 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-05-08T22:38:11.884 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.884 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.884 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.884 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.884 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.884 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.884 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.887 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.887 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.887 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.887 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.887 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.887 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.887 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.887 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.887 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.887 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-05-08T22:38:11.887 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-05-08T22:38:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.889 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.890 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.890 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.890 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.890 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-05-08T22:38:11.890 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-05-08T22:38:11.890 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.890 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.890 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.890 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.890 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.890 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-05-08T22:38:11.891 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.892 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.893 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-05-08T22:38:11.894 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.895 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.895 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.895 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.895 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.895 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.895 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.895 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.895 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.895 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.895 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.895 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.898 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.899 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.900 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.901 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.902 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.902 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.902 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.902 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.902 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.902 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.902 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.902 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.902 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.902 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.902 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.903 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.904 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.905 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.906 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.906 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.906 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.906 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.906 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.906 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.906 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.906 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.906 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.906 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.906 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.907 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.907 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.907 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.907 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.907 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.907 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.907 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.907 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.907 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.907 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.907 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.908 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.909 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.910 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.910 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.910 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.910 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.910 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.910 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.910 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.910 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.910 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.910 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.910 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-05-08T22:38:11.911 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.912 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.915 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.916 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.917 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.917 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.917 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.917 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.917 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.917 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.917 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.917 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.917 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.917 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.917 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.918 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-05-08T22:38:11.918 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-05-08T22:38:11.918 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.918 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.918 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.918 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-05-08T22:38:11.918 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.918 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.918 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.918 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-05-08T22:38:11.920 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-05-08T22:38:11.921 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.922 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.924 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-05-08T22:38:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-05-08T22:38:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-05-08T22:38:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.926 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.926 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.926 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.926 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.926 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.926 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.926 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.926 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.926 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.926 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.926 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.927 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-05-08T22:38:11.928 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-05-08T22:38:11.929 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-05-08T22:38:11.930 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.930 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.930 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.930 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.930 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.930 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.930 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-05-08T22:38:11.930 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-05-08T22:38:11.930 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-05-08T22:38:11.930 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.930 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-05-08T22:38:11.931 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.932 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.933 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.934 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.934 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-05-08T22:38:11.934 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-05-08T22:38:11.934 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-05-08T22:38:11.934 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-05-08T22:38:11.934 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-05-08T22:38:11.934 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T22:38:11.934 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-05-08T22:38:11.934 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-05-08T22:38:11.934 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-05-08T22:38:11.934 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: "services": { 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard": "https://172.21.15.62:8443/" 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: "always_on_modules": { 2024-05-08T22:38:11.935 INFO:teuthology.orchestra.run.smithi062.stdout: "octopus": [ 2024-05-08T22:38:11.936 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-05-08T22:38:11.936 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-05-08T22:38:11.936 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-05-08T22:38:11.936 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-05-08T22:38:11.936 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-05-08T22:38:11.936 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-05-08T22:38:11.936 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-05-08T22:38:11.936 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-05-08T22:38:11.936 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-05-08T22:38:11.936 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-05-08T22:38:11.936 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.937 INFO:teuthology.orchestra.run.smithi062.stdout: "pacific": [ 2024-05-08T22:38:11.937 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-05-08T22:38:11.937 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-05-08T22:38:11.937 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-05-08T22:38:11.937 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-05-08T22:38:11.937 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-05-08T22:38:11.937 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-05-08T22:38:11.937 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-05-08T22:38:11.937 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-05-08T22:38:11.937 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-05-08T22:38:11.937 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-05-08T22:38:11.938 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.938 INFO:teuthology.orchestra.run.smithi062.stdout: "quincy": [ 2024-05-08T22:38:11.938 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-05-08T22:38:11.938 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-05-08T22:38:11.938 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-05-08T22:38:11.938 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-05-08T22:38:11.938 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-05-08T22:38:11.938 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-05-08T22:38:11.938 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-05-08T22:38:11.938 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-05-08T22:38:11.938 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-05-08T22:38:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-05-08T22:38:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-05-08T22:38:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: "reef": [ 2024-05-08T22:38:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-05-08T22:38:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-05-08T22:38:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-05-08T22:38:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-05-08T22:38:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-05-08T22:38:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-05-08T22:38:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-05-08T22:38:11.939 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: "last_failure_osd_epoch": 3, 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: "active_clients": [ 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-05-08T22:38:11.940 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-05-08T22:38:11.941 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 3870206031 2024-05-08T22:38:11.941 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.941 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-05-08T22:38:11.941 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.941 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.941 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "libcephsqlite", 2024-05-08T22:38:11.941 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-05-08T22:38:11.941 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.941 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-05-08T22:38:11.941 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-05-08T22:38:11.941 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 3990867784 2024-05-08T22:38:11.942 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.942 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-05-08T22:38:11.942 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.942 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.942 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-05-08T22:38:11.942 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-05-08T22:38:11.942 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.942 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-05-08T22:38:11.942 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-05-08T22:38:11.942 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 1829209995 2024-05-08T22:38:11.942 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 4274876935 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-05-08T22:38:11.943 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:11.944 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-05-08T22:38:11.944 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:38:11.945 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:10 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/927609863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:38:11.945 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:10 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/289232968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:38:11.946 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:10 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/927609863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:38:11.946 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:10 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/289232968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:38:11.946 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:11 smithi079 ceph-mon[46738]: pgmap v314: 97 pgs: 97 active+clean; 1.9 GiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 1.36k op/s 2024-05-08T22:38:11.946 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:11 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1951164789' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T22:38:11.946 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:11 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:38:11.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.612+0000 7fba2597b700 1 -- 172.21.15.62:0/1951164789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba080a1120 msgr2=0x7fba080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.612+0000 7fba2597b700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba080a1120 0x7fba080a35e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba1000b2e0 tx=0x7fba10005fb0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.612+0000 7fba2597b700 1 -- 172.21.15.62:0/1951164789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba20104290 msgr2=0x7fba201972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:11.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.612+0000 7fba2597b700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba20104290 0x7fba201972f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fba1400d950 tx=0x7fba1400dd10 comp rx=0 tx=0).stop 2024-05-08T22:38:11.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.613+0000 7fba2597b700 1 -- 172.21.15.62:0/1951164789 shutdown_connections 2024-05-08T22:38:11.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.613+0000 7fba2597b700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba20106bc0 0x7fba2019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.613+0000 7fba2597b700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fba080a1120 0x7fba080a35e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.613+0000 7fba2597b700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba20101960 0x7fba20196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.613+0000 7fba2597b700 1 --2- 172.21.15.62:0/1951164789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba20104290 0x7fba201972f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:11.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.613+0000 7fba2597b700 1 -- 172.21.15.62:0/1951164789 >> 172.21.15.62:0/1951164789 conn(0x7fba200fb310 msgr2=0x7fba20102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:11.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.613+0000 7fba2597b700 1 -- 172.21.15.62:0/1951164789 shutdown_connections 2024-05-08T22:38:11.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:10.613+0000 7fba2597b700 1 -- 172.21.15.62:0/1951164789 wait complete. 2024-05-08T22:38:11.953 DEBUG:tasks.check_counter:Getting stats from a 2024-05-08T22:38:11.954 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-08T22:38:11.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:10 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/927609863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:38:11.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:10 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/289232968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:38:11.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:11 smithi062 ceph-mon[42571]: pgmap v314: 97 pgs: 97 active+clean; 1.9 GiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 1.36k op/s 2024-05-08T22:38:11.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:11 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1951164789' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T22:38:11.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:11 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:38:12.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:11 smithi066 ceph-mon[46830]: pgmap v314: 97 pgs: 97 active+clean; 1.9 GiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 1.36k op/s 2024-05-08T22:38:12.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:11 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1951164789' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T22:38:12.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:11 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T22:38:12.183 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:38:12.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:12 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:38:12.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:12 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:38:12.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:12 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:38:12.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:12 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:38:12.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:12 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:38:12.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:12 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:38:13.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:12 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T22:38:13.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:12 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T22:38:13.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:12 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' 2024-05-08T22:38:13.210 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:38:13.210 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:13.210 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 137, 2024-05-08T22:38:13.210 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 129, 2024-05-08T22:38:13.210 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 502646, 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 109673, 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 14, 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.018448546, 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.005026948, 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.014264244, 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.001279567, 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 128, 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.011953884, 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000093389 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.211 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 34, 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000006469, 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000190 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 476464, 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 37072 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 127, 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 191, 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 82726, 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 245212, 2024-05-08T22:38:13.212 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 9, 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 5, 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.018151373, 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.007292773, 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.009956460, 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.001207235, 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 192, 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.028291412, 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000147351 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 243, 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000054659, 2024-05-08T22:38:13.213 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000224 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 149296 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 32, 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 36, 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 25886, 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 29981, 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 8, 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.003624524, 2024-05-08T22:38:13.214 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000585107, 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.002722007, 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000326527, 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 32, 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.045071386, 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001408480 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 21, 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000003728, 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000177 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.215 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 55, 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.001215327, 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000022096 2024-05-08T22:38:13.216 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2, 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000011086, 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000005543 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 3, 2024-05-08T22:38:13.217 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 2, 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2, 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.002218605, 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001109302 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 1, 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 11, 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-05-08T22:38:13.218 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 18, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 18, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 15, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 1, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 1, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 3, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 6, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 3, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 2, 2024-05-08T22:38:13.219 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 1, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 3, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 4, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 234, 2024-05-08T22:38:13.220 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 2, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:38:13.221 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 2, 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 1 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 3, 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-05-08T22:38:13.222 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:38:13.223 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 4, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 4, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 1, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 1, 2024-05-08T22:38:13.224 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 4199810, 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4, 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.008066201, 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002016550 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 20, 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 20, 2024-05-08T22:38:13.225 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 0, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 17, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 18, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 1, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 18, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 18, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 0, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 1, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 1, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 44128, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 198964 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-05-08T22:38:13.226 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 6, 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 3, 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 2, 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 1, 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2, 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.002218605, 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001109302 2024-05-08T22:38:13.227 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.228 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.229 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.230 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.231 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.233 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.234 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.235 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-05-08T22:38:13.236 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.237 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 1, 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 0, 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2024-05-08T22:38:13.238 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 1, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 1, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 114, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:13.239 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:13.240 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:13.241 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 254423, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 82, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 7528, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 158, 2024-05-08T22:38:13.242 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:13.243 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:13.243 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:13.243 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-05-08T22:38:13.243 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 78730, 2024-05-08T22:38:13.243 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 368, 2024-05-08T22:38:13.243 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:13.243 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-05-08T22:38:13.243 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:13.243 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-05-08T22:38:13.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 41, 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 16077, 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 33, 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 41, 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 41, 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.247996211, 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.006048688 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-05-08T22:38:13.246 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 41, 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 64 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 41, 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 0, 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 41, 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 0, 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 10, 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 0, 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 4, 2024-05-08T22:38:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 7, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 6, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 20, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-05-08T22:38:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-05-08T22:38:13.249 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 81, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 2, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 9, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 9, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-05-08T22:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 16, 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 0, 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 1 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-05-08T22:38:13.251 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 292, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 576992, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 292, 2024-05-08T22:38:13.252 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 292, 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 576992, 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:13.253 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 41, 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 16077, 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 41, 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 16077, 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.254 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 41, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 41, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 41, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 41, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.255 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-05-08T22:38:13.256 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x55f4b7298000": { 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 4, 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 5506, 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 4, 2024-05-08T22:38:13.257 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-05-08T22:38:13.258 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-05-08T22:38:13.258 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 4, 2024-05-08T22:38:13.258 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 5506, 2024-05-08T22:38:13.258 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:13.258 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:13.258 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:13.258 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:13.258 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:13.258 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:13.258 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:38:13.661 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-08T22:38:13.661 INFO:tasks.check_counter:Daemon mds.a mds.imported=1 2024-05-08T22:38:13.661 DEBUG:tasks.check_counter:Getting stats from d 2024-05-08T22:38:13.661 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-08T22:38:13.902 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:38:14.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:13 smithi062 ceph-mon[42571]: pgmap v315: 97 pgs: 97 active+clean; 2.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 429 op/s 2024-05-08T22:38:14.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:13 smithi066 ceph-mon[46830]: pgmap v315: 97 pgs: 97 active+clean; 2.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 429 op/s 2024-05-08T22:38:14.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:13 smithi079 ceph-mon[46738]: pgmap v315: 97 pgs: 97 active+clean; 2.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 429 op/s 2024-05-08T22:38:15.002 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:38:15.002 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:15.002 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.001532358, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000424558, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:15.003 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 5, 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 46, 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3104, 2024-05-08T22:38:15.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 84256, 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 3, 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.004480647, 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.002355105, 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001599829, 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 45, 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.002791393, 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000062030 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:15.005 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 84256 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 63, 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 60, 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 427664, 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 32864, 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-05-08T22:38:15.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.009953006, 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.002915466, 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.007979425, 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.001055585, 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 59, 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.006858429, 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000116244 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:15.007 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 427664, 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 32864 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:15.008 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:15.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:15.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148840, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 28, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:15.011 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 64, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 418803, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 64, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 64, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 418803, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:15.013 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:15.014 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:15.014 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:15.014 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:15.014 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:38:15.390 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-08T22:38:15.390 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-08T22:38:15.390 DEBUG:tasks.check_counter:Getting stats from g 2024-05-08T22:38:15.390 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-08T22:38:15.625 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:38:16.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:15 smithi062 ceph-mon[42571]: pgmap v316: 97 pgs: 97 active+clean; 2.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 293 op/s 2024-05-08T22:38:16.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:15 smithi062 ceph-mon[42571]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:38:16.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:15 smithi066 ceph-mon[46830]: pgmap v316: 97 pgs: 97 active+clean; 2.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 293 op/s 2024-05-08T22:38:16.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:15 smithi066 ceph-mon[46830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:38:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:15 smithi079 ceph-mon[46738]: pgmap v316: 97 pgs: 97 active+clean; 2.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 293 op/s 2024-05-08T22:38:16.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:15 smithi079 ceph-mon[46738]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:38:16.617 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:38:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 186, 2024-05-08T22:38:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 186, 2024-05-08T22:38:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 64041, 2024-05-08T22:38:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 57051, 2024-05-08T22:38:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-05-08T22:38:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.020616343, 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.007428455, 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.005131011, 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.005173566, 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 185, 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.009456322, 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000051115 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 376, 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 369, 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 550184, 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 127184, 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.030216279, 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.008627021, 2024-05-08T22:38:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.014082572, 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.008405529, 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 368, 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.011852414, 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000032207 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 466688, 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 32272 2024-05-08T22:38:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 4, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 46, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1508, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 149968, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.004585492, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.002185402, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001876010, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000058531, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 44, 2024-05-08T22:38:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.003532261, 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000080278 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 149024 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-05-08T22:38:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 852, 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.020524866, 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000024090 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-05-08T22:38:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 6, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 2, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 4, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 3, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 3, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-05-08T22:38:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:38:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:38:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-05-08T22:38:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:38:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 4, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-05-08T22:38:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 1, 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 4199810, 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 4199749, 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 4 2024-05-08T22:38:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 8, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 8, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 0, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 6, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 6, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 0, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 6, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 6, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 0, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-05-08T22:38:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 43972, 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 207156 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-05-08T22:38:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-05-08T22:38:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:38:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 1, 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 0, 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-05-08T22:38:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 1571, 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 197337, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 53, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 7528, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 158, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 29549, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 100, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 490, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 21, 2024-05-08T22:38:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 3, 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 490, 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 490, 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.237996358, 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000485706 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 490, 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 491 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 490, 2024-05-08T22:38:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 490, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 300, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 188, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-05-08T22:38:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-05-08T22:38:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-05-08T22:38:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 81, 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 2, 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 6, 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 6, 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-05-08T22:38:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-05-08T22:38:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 562, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 558602, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 562, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 562, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 558602, 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-05-08T22:38:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-05-08T22:38:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-05-08T22:38:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-05-08T22:38:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 490, 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 5511, 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 490, 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 5511, 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 490, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 490, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 490, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 490, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-05-08T22:38:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x560842a8a0a0": { 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-05-08T22:38:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-05-08T22:38:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-05-08T22:38:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-05-08T22:38:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:16.662 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:38:17.052 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-08T22:38:17.052 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-08T22:38:17.052 DEBUG:tasks.check_counter:Getting stats from j 2024-05-08T22:38:17.053 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-08T22:38:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T22:38:18.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:17 smithi066 ceph-mon[46830]: pgmap v317: 97 pgs: 97 active+clean; 1.9 GiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 999 KiB/s wr, 278 op/s 2024-05-08T22:38:18.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:17 smithi062 ceph-mon[42571]: pgmap v317: 97 pgs: 97 active+clean; 1.9 GiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 999 KiB/s wr, 278 op/s 2024-05-08T22:38:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:17 smithi079 ceph-mon[46738]: pgmap v317: 97 pgs: 97 active+clean; 1.9 GiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 999 KiB/s wr, 278 op/s 2024-05-08T22:38:18.284 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-05-08T22:38:18.284 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:18.284 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 4, 2024-05-08T22:38:18.284 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 4, 2024-05-08T22:38:18.284 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 2944, 2024-05-08T22:38:18.284 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 448, 2024-05-08T22:38:18.284 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-05-08T22:38:18.284 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.001625489, 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000038181, 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000544152, 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3, 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000711759, 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000237253 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:18.285 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 64, 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 61, 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 427936, 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 33456, 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.011235554, 2024-05-08T22:38:18.286 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.003422315, 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.009349024, 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.001059120, 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 60, 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.002708156, 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000045135 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:18.287 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 427936, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 33456 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 45, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 160, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 83984, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 3, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.004479896, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.002153011, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001830519, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:18.288 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 43, 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.002004544, 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000046617 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 83984 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-05-08T22:38:18.289 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:18.290 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:18.291 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148863, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 28, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-05-08T22:38:18.292 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-05-08T22:38:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 65, 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 418933, 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 65, 2024-05-08T22:38:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-05-08T22:38:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-05-08T22:38:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 65, 2024-05-08T22:38:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 418933, 2024-05-08T22:38:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-05-08T22:38:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-05-08T22:38:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-05-08T22:38:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-05-08T22:38:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-05-08T22:38:18.295 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-05-08T22:38:18.670 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-08T22:38:18.670 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-08T22:38:18.670 DEBUG:tasks.check_counter:Getting stats from b 2024-05-08T22:38:18.670 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-08T22:38:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:38:19.853 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-05-08T22:38:19.853 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:19.854 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 717, 2024-05-08T22:38:19.854 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 545, 2024-05-08T22:38:19.854 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 725057, 2024-05-08T22:38:19.854 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 297329, 2024-05-08T22:38:19.854 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 14, 2024-05-08T22:38:19.854 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-05-08T22:38:19.854 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.071097647, 2024-05-08T22:38:19.854 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.018944841, 2024-05-08T22:38:19.854 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.041348279, 2024-05-08T22:38:19.854 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.015805600, 2024-05-08T22:38:19.854 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 546, 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.155885533, 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000285504 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 744, 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000149106, 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000200 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 505168, 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 38832 2024-05-08T22:38:19.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 464, 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 432, 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 184771, 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 342354, 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 10, 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 6, 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.041490570, 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.012676306, 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.017945959, 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.010003752, 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:19.856 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 431, 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.218592361, 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000507174 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 421, 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000083450, 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000198 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 153808 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:19.857 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 568, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 459, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 220420, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 491345, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 27, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 8, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.053132274, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.012460368, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.024837902, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.014381080, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 455, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.334401588, 2024-05-08T22:38:19.858 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000734948 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 384, 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000075724, 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000197 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 54192, 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 31744 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-05-08T22:38:19.859 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 204, 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.036124048, 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000177078 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2, 2024-05-08T22:38:19.860 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000023179, 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000011589 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 201, 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 200, 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 200, 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.086550674, 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000432753 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-05-08T22:38:19.861 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 1, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 26, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 19, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 2, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 4, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 13, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 16, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 5, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 2, 2024-05-08T22:38:19.862 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 2, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 3, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 379, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 372, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 1, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 0, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 248, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 2, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 4, 2024-05-08T22:38:19.863 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 1, 2024-05-08T22:38:19.864 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 1, 2024-05-08T22:38:19.864 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-05-08T22:38:19.864 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:38:19.864 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-05-08T22:38:19.864 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 4, 2024-05-08T22:38:19.864 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 234, 2024-05-08T22:38:19.864 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-05-08T22:38:19.864 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:38:19.864 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-05-08T22:38:19.864 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:38:19.864 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 16, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 10, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 22, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 2, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2024-05-08T22:38:19.865 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 16 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 0, 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 0, 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 0, 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 7, 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2024-05-08T22:38:19.866 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 2, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-05-08T22:38:19.867 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 52, 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 46, 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 46, 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 6, 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 3, 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 9, 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 8, 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 8, 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 2, 2024-05-08T22:38:19.868 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 1, 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4321065, 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 4336204, 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 0, 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 42, 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.082341302, 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001960507 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 0 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-05-08T22:38:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 20, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 24, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 4, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 17, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 19, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 2, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 19, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 24, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 5, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 2, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 14, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 12, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 50468, 2024-05-08T22:38:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 207156 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 216, 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 3, 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 201, 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 9, 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 6, 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 4, 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.002800085, 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000700021 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-05-08T22:38:19.871 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 178, 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.048586791, 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000272959 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1, 2024-05-08T22:38:19.873 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000137741, 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000137741 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.874 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 7, 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.011497747, 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001642535 2024-05-08T22:38:19.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.876 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2, 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.002335655, 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001167827 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.877 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.878 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1, 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001492861, 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001492861 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.879 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 7, 2024-05-08T22:38:19.880 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.019699794, 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.002814256 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:38:19.881 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 2, 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 1, 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 1, 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 68, 2024-05-08T22:38:19.882 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 68, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 121, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 120, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 120, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:19.883 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:19.884 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 322729, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 130, 2024-05-08T22:38:19.885 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 352, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 4, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7528, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 158, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:19.886 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 82141, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 506, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 647, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 175068, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 37, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 647, 2024-05-08T22:38:19.887 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 647, 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 17.579863560, 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.027171350 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 647, 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 727 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 647, 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 0, 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 647, 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-05-08T22:38:19.888 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 9, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 22, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 0, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 42, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 31, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 506, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-05-08T22:38:19.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 44, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-05-08T22:38:19.890 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-05-08T22:38:19.891 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 81, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 3, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 12, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 12, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 66, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 0, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 7 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 1, 2024-05-08T22:38:19.892 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-05-08T22:38:19.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 1745, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 958384, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 1745, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 1745, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 958384, 2024-05-08T22:38:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:19.895 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 647, 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 175068, 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 647, 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 175068, 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:19.896 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 647, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 647, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 647, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 647, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.897 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:19.898 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x560b7a774000": { 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 52, 2024-05-08T22:38:19.899 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 141900, 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 52, 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 42, 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 141900, 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:19.900 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-05-08T22:38:20.000 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:19 smithi066 ceph-mon[46830]: pgmap v318: 97 pgs: 97 active+clean; 955 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 293 op/s 2024-05-08T22:38:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:19 smithi079 ceph-mon[46738]: pgmap v318: 97 pgs: 97 active+clean; 955 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 293 op/s 2024-05-08T22:38:20.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:19 smithi062 ceph-mon[42571]: pgmap v318: 97 pgs: 97 active+clean; 955 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 293 op/s 2024-05-08T22:38:20.240 INFO:tasks.check_counter:Daemon mds.b mds.exported=2 2024-05-08T22:38:20.240 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-05-08T22:38:20.240 DEBUG:tasks.check_counter:Getting stats from e 2024-05-08T22:38:20.240 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-08T22:38:20.470 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:38:21.420 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-05-08T22:38:21.420 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:21.420 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 524, 2024-05-08T22:38:21.420 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 526, 2024-05-08T22:38:21.420 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 12509022, 2024-05-08T22:38:21.420 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 162971, 2024-05-08T22:38:21.420 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 6, 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 2, 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.051472530, 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.013358682, 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.018774425, 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.014296647, 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 524, 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.018680776, 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000035650 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.421 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 83, 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 73, 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 485038, 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 36257, 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-05-08T22:38:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.014515211, 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.004064381, 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.011559312, 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.001547398, 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 72, 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.006615817, 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000091886 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 483744, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 34640 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 6, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 48, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 3372, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 151703, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.003833610, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.001783075, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.001572730, 2024-05-08T22:38:21.424 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000020503, 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 47, 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.004170544, 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000088734 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 151392 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.425 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 924, 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.022189356, 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000024014 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-05-08T22:38:21.426 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 0, 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.427 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 0, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 0, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 1217, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 0, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 1217, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 1217, 2024-05-08T22:38:21.428 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 1, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-05-08T22:38:21.429 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-05-08T22:38:21.430 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:38:21.431 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 0, 2024-05-08T22:38:21.432 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:38:21.432 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2024-05-08T22:38:21.432 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 0 2024-05-08T22:38:21.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.432 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-05-08T22:38:21.432 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2024-05-08T22:38:21.432 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 0, 2024-05-08T22:38:21.432 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:38:21.433 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 0, 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-05-08T22:38:21.434 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 6070, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 0, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 7, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 16553667, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 16553068, 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-05-08T22:38:21.435 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 6070 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 8, 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 1224, 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 1216, 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 6, 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 17, 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 11, 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 1217, 2024-05-08T22:38:21.436 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 1229, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 12, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 0, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 0, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 0, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 50660, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 207156 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 0, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 0, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 0, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 0, 2024-05-08T22:38:21.437 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.438 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-05-08T22:38:21.439 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.440 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.442 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-05-08T22:38:21.442 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.444 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-05-08T22:38:21.444 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.444 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.444 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.444 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-05-08T22:38:21.444 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.444 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.444 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.444 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.446 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.447 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-05-08T22:38:21.449 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 2, 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 1, 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 2, 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 0, 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 0, 2024-05-08T22:38:21.450 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 2826, 2024-05-08T22:38:21.451 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-05-08T22:38:21.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.451 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-05-08T22:38:21.451 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:38:21.451 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-05-08T22:38:21.451 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:21.451 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:21.451 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:21.451 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:21.451 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:21.452 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:21.452 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:21.452 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:21.452 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:21.452 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:21.452 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:21.452 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:21.452 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:21.452 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:21.453 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 220169, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 1191, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-05-08T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7528, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 158, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 3502088, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 112571, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:21.455 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 530, 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 21, 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 3, 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 530, 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 530, 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.308997602, 2024-05-08T22:38:21.456 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000583014 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 530, 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 531 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 530, 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 530, 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 0, 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 313, 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 0, 2024-05-08T22:38:21.457 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 215, 2024-05-08T22:38:21.458 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 0, 2024-05-08T22:38:21.458 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 0, 2024-05-08T22:38:21.458 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-05-08T22:38:21.458 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-05-08T22:38:21.458 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-05-08T22:38:21.458 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-05-08T22:38:21.458 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 0, 2024-05-08T22:38:21.458 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-05-08T22:38:21.458 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-05-08T22:38:21.458 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-05-08T22:38:21.458 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 1, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-05-08T22:38:21.459 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-05-08T22:38:21.460 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 81, 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 5, 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 5, 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 0, 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 2, 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 0 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-05-08T22:38:21.461 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 0, 2024-05-08T22:38:21.462 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:38:21.462 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-05-08T22:38:21.462 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-05-08T22:38:21.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.462 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-05-08T22:38:21.462 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-05-08T22:38:21.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.462 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-05-08T22:38:21.462 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-05-08T22:38:21.462 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:38:21.462 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 609, 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 12935712, 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:21.463 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 609, 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 609, 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 12935712, 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 530, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 12359368, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 530, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 12359368, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-05-08T22:38:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-05-08T22:38:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 530, 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 530, 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 530, 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 530, 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x55f1254220a0": { 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-05-08T22:38:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:21.470 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:21.470 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:21.470 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:21.470 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:21.470 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:21.470 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-05-08T22:38:21.797 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-08T22:38:21.797 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-08T22:38:21.797 DEBUG:tasks.check_counter:Getting stats from h 2024-05-08T22:38:21.798 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-08T22:38:22.031 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:38:22.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:21 smithi066 ceph-mon[46830]: pgmap v319: 97 pgs: 97 active+clean; 12 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 513 KiB/s wr, 235 op/s 2024-05-08T22:38:22.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:21 smithi079 ceph-mon[46738]: pgmap v319: 97 pgs: 97 active+clean; 12 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 513 KiB/s wr, 235 op/s 2024-05-08T22:38:22.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:21 smithi062 ceph-mon[42571]: pgmap v319: 97 pgs: 97 active+clean; 12 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 513 KiB/s wr, 235 op/s 2024-05-08T22:38:23.264 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-05-08T22:38:23.265 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:23.265 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 104, 2024-05-08T22:38:23.265 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 150, 2024-05-08T22:38:23.265 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 85254, 2024-05-08T22:38:23.265 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 90504, 2024-05-08T22:38:23.266 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 9, 2024-05-08T22:38:23.266 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 2, 2024-05-08T22:38:23.266 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.017252758, 2024-05-08T22:38:23.266 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.005915491, 2024-05-08T22:38:23.266 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.008081676, 2024-05-08T22:38:23.266 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.001255467, 2024-05-08T22:38:23.266 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:23.266 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 153, 2024-05-08T22:38:23.267 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.035050966, 2024-05-08T22:38:23.267 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000229091 2024-05-08T22:38:23.267 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.267 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:23.267 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 202, 2024-05-08T22:38:23.267 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000058098, 2024-05-08T22:38:23.267 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000287 2024-05-08T22:38:23.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T22:38:23.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T22:38:23.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.268 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:23.268 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 27, 2024-05-08T22:38:23.269 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 71, 2024-05-08T22:38:23.269 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 30255, 2024-05-08T22:38:23.269 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 184903, 2024-05-08T22:38:23.269 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 8, 2024-05-08T22:38:23.269 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 5, 2024-05-08T22:38:23.269 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.006687502, 2024-05-08T22:38:23.270 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.002211993, 2024-05-08T22:38:23.270 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.003954680, 2024-05-08T22:38:23.270 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000273516, 2024-05-08T22:38:23.270 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:23.270 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 69, 2024-05-08T22:38:23.270 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.018947045, 2024-05-08T22:38:23.270 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000274594 2024-05-08T22:38:23.270 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 23, 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000018228, 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000792 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 152512 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 127, 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 120, 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 519496, 2024-05-08T22:38:23.271 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 112990, 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 12, 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 6, 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.019705495, 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.004811414, 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.017235186, 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.001609099, 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 115, 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.013131787, 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000114189 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:23.272 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 27, 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000005068, 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000187 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 490448, 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 38480 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 35, 2024-05-08T22:38:23.273 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001705695, 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000048734 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2, 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000015726, 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000007863 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.274 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 0, 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 0, 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 11, 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-05-08T22:38:23.275 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 10, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 0, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 10, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 10, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 0, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 0, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 0, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 1, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 0, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 0, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 0, 2024-05-08T22:38:23.276 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 0, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 2000, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 2, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-05-08T22:38:23.277 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 4, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 234, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:38:23.278 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 0, 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 0 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 0, 2024-05-08T22:38:23.279 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 0, 2024-05-08T22:38:23.280 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 1, 2024-05-08T22:38:23.280 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 0, 2024-05-08T22:38:23.280 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-05-08T22:38:23.280 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-05-08T22:38:23.280 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:38:23.280 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2024-05-08T22:38:23.280 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2024-05-08T22:38:23.280 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-05-08T22:38:23.280 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-05-08T22:38:23.280 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-05-08T22:38:23.280 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:38:23.281 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:38:23.281 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-05-08T22:38:23.281 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-05-08T22:38:23.281 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:38:23.281 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2024-05-08T22:38:23.281 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-05-08T22:38:23.281 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:38:23.282 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-05-08T22:38:23.282 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-05-08T22:38:23.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-05-08T22:38:23.282 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 1, 2024-05-08T22:38:23.286 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-05-08T22:38:23.286 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-05-08T22:38:23.286 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 1, 2024-05-08T22:38:23.286 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-05-08T22:38:23.286 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-05-08T22:38:23.287 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 1, 2024-05-08T22:38:23.287 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-05-08T22:38:23.287 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-05-08T22:38:23.287 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 1, 2024-05-08T22:38:23.287 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-05-08T22:38:23.287 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-05-08T22:38:23.288 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2024-05-08T22:38:23.288 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 4194844, 2024-05-08T22:38:23.288 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 0, 2024-05-08T22:38:23.288 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-05-08T22:38:23.288 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1, 2024-05-08T22:38:23.288 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001331205, 2024-05-08T22:38:23.289 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001331205 2024-05-08T22:38:23.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.289 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 0 2024-05-08T22:38:23.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.289 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-05-08T22:38:23.290 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 13, 2024-05-08T22:38:23.290 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 13, 2024-05-08T22:38:23.290 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 0, 2024-05-08T22:38:23.290 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 11, 2024-05-08T22:38:23.290 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 12, 2024-05-08T22:38:23.290 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 1, 2024-05-08T22:38:23.291 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 10, 2024-05-08T22:38:23.291 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 10, 2024-05-08T22:38:23.291 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 0, 2024-05-08T22:38:23.291 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 0, 2024-05-08T22:38:23.291 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 0, 2024-05-08T22:38:23.291 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 0, 2024-05-08T22:38:23.292 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 44552, 2024-05-08T22:38:23.292 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 198964 2024-05-08T22:38:23.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.292 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-05-08T22:38:23.292 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 0, 2024-05-08T22:38:23.293 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2024-05-08T22:38:23.293 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 0, 2024-05-08T22:38:23.293 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 0, 2024-05-08T22:38:23.293 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 0, 2024-05-08T22:38:23.293 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-05-08T22:38:23.293 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.294 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.294 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.294 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-05-08T22:38:23.294 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.294 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.295 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.295 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-05-08T22:38:23.295 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.295 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.296 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.296 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.296 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-05-08T22:38:23.296 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.296 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.296 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.297 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-05-08T22:38:23.297 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.297 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.297 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.298 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-05-08T22:38:23.298 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.298 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.298 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.299 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-05-08T22:38:23.299 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.299 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.299 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.299 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-05-08T22:38:23.300 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.300 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.300 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.300 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-05-08T22:38:23.300 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.300 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.300 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.300 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.302 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-05-08T22:38:23.302 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.302 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.302 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.302 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-05-08T22:38:23.302 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.302 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.302 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.303 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-05-08T22:38:23.303 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.303 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.303 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.303 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-05-08T22:38:23.303 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.303 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.303 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.304 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-05-08T22:38:23.304 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.304 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.304 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.305 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-05-08T22:38:23.305 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.305 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.305 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.306 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-05-08T22:38:23.306 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.306 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.306 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.306 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-05-08T22:38:23.306 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.306 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.307 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.307 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-05-08T22:38:23.307 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.307 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.307 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.307 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-05-08T22:38:23.308 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.308 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.308 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.308 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-05-08T22:38:23.308 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.308 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.308 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.309 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-05-08T22:38:23.309 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.309 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.309 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.309 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-05-08T22:38:23.309 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.309 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.310 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.310 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-05-08T22:38:23.310 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.310 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.310 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.311 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-05-08T22:38:23.311 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.311 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.311 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.311 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-05-08T22:38:23.311 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.312 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.312 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.312 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-05-08T22:38:23.312 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.312 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.312 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.312 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:38:23.312 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 0, 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 0, 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 0, 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 0, 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-05-08T22:38:23.313 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 0, 2024-05-08T22:38:23.314 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 0, 2024-05-08T22:38:23.314 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 0, 2024-05-08T22:38:23.314 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-05-08T22:38:23.314 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.314 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-05-08T22:38:23.314 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:38:23.314 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-05-08T22:38:23.314 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:23.314 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:23.315 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:23.316 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 221591, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 66, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:23.317 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7528, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 158, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 50032, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 197, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:23.318 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 35, 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 10662, 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 27, 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 35, 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 35, 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.206998413, 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.005914240 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-05-08T22:38:23.319 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 35, 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 57 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 35, 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 0, 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 35, 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 0, 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 10, 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 0, 2024-05-08T22:38:23.320 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 1, 2024-05-08T22:38:23.321 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 5, 2024-05-08T22:38:23.321 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-05-08T22:38:23.321 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-05-08T22:38:23.321 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-05-08T22:38:23.321 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-05-08T22:38:23.321 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 6, 2024-05-08T22:38:23.321 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-05-08T22:38:23.321 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-05-08T22:38:23.321 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-05-08T22:38:23.322 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 0, 2024-05-08T22:38:23.322 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 20, 2024-05-08T22:38:23.322 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:38:23.322 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-05-08T22:38:23.322 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:38:23.322 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-05-08T22:38:23.322 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-05-08T22:38:23.322 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-05-08T22:38:23.322 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:38:23.322 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-05-08T22:38:23.323 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 81, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 8, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 8, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 14, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 0, 2024-05-08T22:38:23.324 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 1 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 1, 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:38:23.325 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 254, 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 604092, 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:23.326 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 254, 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 254, 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 604092, 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-05-08T22:38:23.327 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 35, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 10662, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 35, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 10662, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.328 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 35, 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 35, 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 35, 2024-05-08T22:38:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 35, 2024-05-08T22:38:23.330 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:23.330 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.330 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.330 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.330 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:23.330 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.330 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:38:23.330 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:23.330 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-08T22:38:23.330 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:23.331 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:23.331 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:23.331 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:23.331 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:23.331 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:23.331 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:23.331 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-05-08T22:38:23.331 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-05-08T22:38:23.331 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:23.331 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.331 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x559c6bbd2000": { 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 1, 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 540, 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 1, 2024-05-08T22:38:23.332 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:23.333 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:23.333 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 1, 2024-05-08T22:38:23.333 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 540, 2024-05-08T22:38:23.333 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:23.333 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:23.333 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:23.333 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:23.333 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:23.333 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:23.333 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-05-08T22:38:23.621 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-08T22:38:23.621 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-08T22:38:23.621 DEBUG:tasks.check_counter:Getting stats from k 2024-05-08T22:38:23.621 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-08T22:38:23.852 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T22:38:23.966 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:23 smithi066 ceph-mon[46830]: pgmap v320: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 449 KiB/s wr, 187 op/s 2024-05-08T22:38:24.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:23 smithi079 ceph-mon[46738]: pgmap v320: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 449 KiB/s wr, 187 op/s 2024-05-08T22:38:24.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:23 smithi062 ceph-mon[42571]: pgmap v320: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 449 KiB/s wr, 187 op/s 2024-05-08T22:38:24.804 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 271, 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 260, 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 554160, 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 94914, 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.036078210, 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.012260738, 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.015395963, 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.007348315, 2024-05-08T22:38:24.805 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 259, 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.016352408, 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000063136 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 490320, 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 35824 2024-05-08T22:38:24.806 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 8, 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 52, 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 4494, 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 154313, 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 4, 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.004915572, 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.002210326, 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.002051484, 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000094254, 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:24.807 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 51, 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.012542878, 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000245938 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 153312 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 193, 2024-05-08T22:38:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 195, 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 51115, 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 60024, 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.016846768, 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.004368830, 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.005465960, 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.005242928, 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 193, 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.007972439, 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000041307 2024-05-08T22:38:24.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-05-08T22:38:24.810 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 765, 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.020148878, 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000026338 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.811 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 0, 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-05-08T22:38:24.812 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 0, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 1, 2024-05-08T22:38:24.813 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 0, 2024-05-08T22:38:24.814 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 0, 2024-05-08T22:38:24.814 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 0, 2024-05-08T22:38:24.814 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 0, 2024-05-08T22:38:24.814 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:38:24.814 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-05-08T22:38:24.814 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-05-08T22:38:24.814 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 0, 2024-05-08T22:38:24.814 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-05-08T22:38:24.814 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-05-08T22:38:24.815 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:38:24.816 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 0 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-05-08T22:38:24.817 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-05-08T22:38:24.818 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-05-08T22:38:24.818 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:38:24.818 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:38:24.818 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-05-08T22:38:24.818 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-05-08T22:38:24.818 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:38:24.818 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2024-05-08T22:38:24.818 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-05-08T22:38:24.818 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 0, 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 1, 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 0, 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-05-08T22:38:24.819 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 1, 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 4194844, 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 4194304, 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 1 2024-05-08T22:38:24.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 3, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 3, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 0, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 1, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 1, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 0, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 0, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 0, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 0, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 0, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 0, 2024-05-08T22:38:24.821 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 0, 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 44848, 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 207156 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 0, 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 0, 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 0, 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 0, 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.823 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.824 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.826 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.827 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.828 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.829 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-05-08T22:38:24.830 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.831 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:24.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 0, 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 0, 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 0, 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 0, 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 0, 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 0, 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 0, 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-05-08T22:38:24.834 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:24.835 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:24.836 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 189239, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 48, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:38:24.837 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7528, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 158, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-05-08T22:38:24.838 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 8320, 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 13, 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 386, 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 21, 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 3, 2024-05-08T22:38:24.839 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 386, 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 386, 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.201998437, 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000523312 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 386, 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 387 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 386, 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 386, 2024-05-08T22:38:24.840 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 190, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 194, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-05-08T22:38:24.841 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 1, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-05-08T22:38:24.842 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-05-08T22:38:24.843 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 81, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 5, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 5, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 0, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 2, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 0 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-05-08T22:38:24.844 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 0, 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-05-08T22:38:24.845 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 468, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 560586, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 468, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:24.846 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 468, 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 560586, 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:24.847 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:24.848 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:24.848 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:24.848 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 386, 2024-05-08T22:38:24.848 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 545, 2024-05-08T22:38:24.848 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 386, 2024-05-08T22:38:24.848 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 545, 2024-05-08T22:38:24.848 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:24.848 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.848 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.848 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 386, 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 386, 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 386, 2024-05-08T22:38:24.849 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 386, 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:24.850 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x5610a401c0a0": { 2024-05-08T22:38:24.851 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-08T22:38:24.852 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:24.853 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-08T22:38:24.853 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-05-08T22:38:25.220 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-08T22:38:25.221 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-08T22:38:25.221 DEBUG:tasks.check_counter:Getting stats from c 2024-05-08T22:38:25.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-08T22:38:25.454 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:38:25.930 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:25 smithi079 ceph-mon[46738]: pgmap v321: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 141 op/s 2024-05-08T22:38:25.930 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:25 smithi079 ceph-mon[46738]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:38:26.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:25 smithi066 ceph-mon[46830]: pgmap v321: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 141 op/s 2024-05-08T22:38:26.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:25 smithi066 ceph-mon[46830]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:38:26.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:25 smithi062 ceph-mon[42571]: pgmap v321: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 141 op/s 2024-05-08T22:38:26.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:25 smithi062 ceph-mon[42571]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T22:38:26.522 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-05-08T22:38:26.522 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:26.522 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 121, 2024-05-08T22:38:26.522 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 168, 2024-05-08T22:38:26.522 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 107722, 2024-05-08T22:38:26.522 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 151046, 2024-05-08T22:38:26.522 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 10, 2024-05-08T22:38:26.522 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 3, 2024-05-08T22:38:26.522 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.018433850, 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.005776972, 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.008710521, 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.001738008, 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 170, 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.026314850, 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000154793 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 230, 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000062765, 2024-05-08T22:38:26.523 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000272 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 26, 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 69, 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 8127, 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 206732, 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 13, 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 3, 2024-05-08T22:38:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.009332887, 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.002735418, 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.005457334, 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000358129, 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 68, 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.014048351, 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000206593 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 9, 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000002033, 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000225 2024-05-08T22:38:26.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 154704 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 132, 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 122, 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 531242, 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 69779, 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 8, 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 7, 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.016738914, 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.004484555, 2024-05-08T22:38:26.526 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.013379268, 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.001534527, 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 117, 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.010662673, 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000091133 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 35, 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000005921, 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000169 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 502976, 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 40512 2024-05-08T22:38:26.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 49, 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.001566449, 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000031968 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:26.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2, 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000015057, 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000007528 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 3, 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 2, 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2, 2024-05-08T22:38:26.529 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000374631, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000187315 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 1, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 11, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 10, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 0, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 10, 2024-05-08T22:38:26.530 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 0, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 10, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 0, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 1, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 1, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 2, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 3, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 2, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 0, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 0, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 0, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-05-08T22:38:26.531 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 0, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 0, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 0, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 0, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 4, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 234, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-05-08T22:38:26.532 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:38:26.533 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 0, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 1 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 0, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 0, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 1, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 0, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 0, 2024-05-08T22:38:26.534 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-05-08T22:38:26.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 4, 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 4, 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 1, 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 1, 2024-05-08T22:38:26.536 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 4194304, 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 4195692, 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 0, 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 4, 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.007342437, 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001835609 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 0 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-05-08T22:38:26.537 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 13, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 13, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 0, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 12, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 12, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 0, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 10, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 10, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 0, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 1, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 2, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 1, 2024-05-08T22:38:26.538 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 44748, 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 198964 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 3, 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 3, 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 6, 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 0, 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1, 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000246927, 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000246927 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.540 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.541 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1, 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000127704, 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000127704 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.542 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-05-08T22:38:26.543 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.544 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-05-08T22:38:26.545 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.547 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.548 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:38:26.550 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:38:26.551 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-05-08T22:38:26.551 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.551 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.551 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:26.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.551 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-05-08T22:38:26.551 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-05-08T22:38:26.551 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 2, 2024-05-08T22:38:26.551 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 1, 2024-05-08T22:38:26.551 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 1, 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 1, 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 0, 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 126, 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:26.552 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:26.553 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-05-08T22:38:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 246599, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 81, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:26.555 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 7528, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 158, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 51824, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 205, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-05-08T22:38:26.556 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 40, 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 11690, 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 27, 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 40, 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 40, 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.214995209, 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.005374880 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 40, 2024-05-08T22:38:26.557 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 62 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 40, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 0, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 40, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 0, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 10, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 0, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 4, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 7, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-05-08T22:38:26.558 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 6, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 20, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-05-08T22:38:26.559 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-05-08T22:38:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 81, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 2, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 8, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 8, 2024-05-08T22:38:26.561 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 0, 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 14, 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 0, 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 1 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 1, 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-05-08T22:38:26.562 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 0 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 0, 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 0, 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-08T22:38:26.563 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 275, 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 613848, 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 275, 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 275, 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 613848, 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:26.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 40, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 11690, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 40, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 11690, 2024-05-08T22:38:26.565 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:26.566 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 40, 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 40, 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 40, 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 40, 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:26.567 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.568 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x5580dff72000": { 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 4, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 1388, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 4, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 4, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 1388, 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:26.569 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:26.570 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:26.570 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:26.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:26.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:26.570 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-05-08T22:38:26.893 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-08T22:38:26.893 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-08T22:38:26.893 DEBUG:tasks.check_counter:Getting stats from f 2024-05-08T22:38:26.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-08T22:38:27.116 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:38:27.888 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:27 smithi079 ceph-mon[46738]: pgmap v322: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 142 op/s 2024-05-08T22:38:28.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:27 smithi066 ceph-mon[46830]: pgmap v322: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 142 op/s 2024-05-08T22:38:28.120 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-05-08T22:38:28.120 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:28.120 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 417, 2024-05-08T22:38:28.120 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 416, 2024-05-08T22:38:28.120 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 267587, 2024-05-08T22:38:28.120 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 128590, 2024-05-08T22:38:28.120 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 6, 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 2, 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.041614246, 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.012588528, 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.011528647, 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.012520628, 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 415, 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.018465204, 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000044494 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.121 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 5, 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 50, 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 1816, 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 153998, 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 5, 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-05-08T22:38:28.122 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.004196015, 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.001748224, 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.001697506, 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000142075, 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 48, 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.012075299, 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000251568 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.123 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 152688 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 255, 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 244, 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 533083, 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 89507, 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 3, 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 2, 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.030026853, 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.009650756, 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.013757895, 2024-05-08T22:38:28.124 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.007284049, 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 243, 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.012239658, 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000050368 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 490208, 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 35392 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.125 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 972, 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.028297201, 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000029112 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-05-08T22:38:28.126 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 0, 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 0, 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.127 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 0, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 0, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 10, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 3, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 7, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 0, 2024-05-08T22:38:28.128 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 6, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 3, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-05-08T22:38:28.129 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 0, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 0, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 0, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 0, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 4, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 234, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-05-08T22:38:28.130 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:38:28.131 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 0, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 0 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 0, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 0, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 0, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 0, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 0, 2024-05-08T22:38:28.132 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.133 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 0, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 52, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 0, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 2, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-05-08T22:38:28.134 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 4321065, 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 4336204, 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 4333371, 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 52 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 11, 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 14, 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 3, 2024-05-08T22:38:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 7, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 9, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 2, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 10, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 14, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 4, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 0, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 0, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 0, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 49960, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 207156 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 0, 2024-05-08T22:38:28.136 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 0, 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 0, 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 0, 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.137 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.138 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.139 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.139 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-05-08T22:38:28.139 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.139 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.139 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.139 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-05-08T22:38:28.139 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.139 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.139 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.140 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-05-08T22:38:28.140 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.140 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.140 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.140 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-05-08T22:38:28.140 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.140 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.140 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.141 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.142 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.144 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-05-08T22:38:28.145 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.147 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.148 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 2, 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 1, 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 2, 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 0, 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 0, 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 2964, 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-05-08T22:38:28.149 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:28.150 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:28.151 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 189279, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 49, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-05-08T22:38:28.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 7528, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 158, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 40053, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 164, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:28.153 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 590, 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 26, 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 4, 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 590, 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-05-08T22:38:28.154 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 590, 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.932979183, 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001581320 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 590, 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 591 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 590, 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 590, 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 0, 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-05-08T22:38:28.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 365, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 0, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 222, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 0, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 0, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 0, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 2, 2024-05-08T22:38:28.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-05-08T22:38:28.157 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-05-08T22:38:28.158 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-05-08T22:38:28.158 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-05-08T22:38:28.158 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-05-08T22:38:28.158 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-05-08T22:38:28.158 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-05-08T22:38:28.158 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-05-08T22:38:28.158 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-05-08T22:38:28.158 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-05-08T22:38:28.158 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-05-08T22:38:28.162 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-05-08T22:38:28.162 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-05-08T22:38:28.162 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-05-08T22:38:28.162 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-05-08T22:38:28.162 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 81, 2024-05-08T22:38:28.162 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-05-08T22:38:28.162 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 2, 2024-05-08T22:38:28.162 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 5, 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 5, 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 0, 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 0, 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 2, 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 0 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 0, 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-05-08T22:38:28.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 0 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 0, 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 0, 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:28.164 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 673, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 734859, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 673, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 673, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 734859, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.165 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 590, 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 141910, 2024-05-08T22:38:28.166 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 590, 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 141910, 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 590, 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 590, 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 590, 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 590, 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:38:28.168 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.169 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x563f53f8e0a0": { 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-05-08T22:38:28.170 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-05-08T22:38:28.171 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:28.171 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:28.171 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:28.171 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:28.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:28.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:28.171 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-05-08T22:38:28.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:27 smithi062 ceph-mon[42571]: pgmap v322: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 142 op/s 2024-05-08T22:38:28.483 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-08T22:38:28.484 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-08T22:38:28.484 DEBUG:tasks.check_counter:Getting stats from i 2024-05-08T22:38:28.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-08T22:38:28.708 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:38:29.686 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-05-08T22:38:29.686 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:29.686 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 2248, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 2270, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 1100765, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 13153881, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 10, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.162447128, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.064328216, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.046630703, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.043586744, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2270, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.079104282, 2024-05-08T22:38:29.687 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000034847 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 286, 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000064862, 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000226 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 507888, 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 39696 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 6993, 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 13152, 2024-05-08T22:38:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 1961233, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 5222226, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 13, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 5, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.374391263, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.160661716, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 2.129743225, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.019341971, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 13150, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.213625733, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000016245 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 16449, 2024-05-08T22:38:29.689 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.002096593, 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000127 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 154816 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 633, 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 635, 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 329799, 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 511085, 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 12, 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 6, 2024-05-08T22:38:29.690 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.037846388, 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.011153535, 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.012593368, 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.011756265, 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 633, 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.042521293, 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000067174 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 51, 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000008405, 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000164 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.691 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2521, 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 1.812455632, 2024-05-08T22:38:29.692 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000718943 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 4365, 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.126822723, 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000029054 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 2434, 2024-05-08T22:38:29.693 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 2434, 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2434, 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 1.823605709, 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000749221 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 0, 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 11, 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-05-08T22:38:29.694 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 14, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 0, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 0, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 14, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 14, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 0, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 1, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 1, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 2, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 2443, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 2431, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 0, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 1, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:38:29.695 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 0, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 1, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 1, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 1, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 1, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 4, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 234, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-05-08T22:38:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 44, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 5924, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:38:29.697 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 2430, 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 1210, 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 1056, 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 1225 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 0, 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 0, 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 2, 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-05-08T22:38:29.698 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 1212, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 1212, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 1, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-05-08T22:38:29.699 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 6070, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 6070, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 7, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 7, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-05-08T22:38:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 4194304, 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 16553667, 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 0, 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1234, 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 1.974854453, 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001600368 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 0 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 17, 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 1229, 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 1212, 2024-05-08T22:38:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 16, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 23, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 7, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 15, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 1234, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 1219, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 1, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 1214, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 1213, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 62660, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 198964 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 2484, 2024-05-08T22:38:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 2434, 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 7, 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 2, 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1206, 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.620740487, 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000514710 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-05-08T22:38:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 5, 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.001537866, 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000307573 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-05-08T22:38:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-05-08T22:38:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 5, 2024-05-08T22:38:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.006200101, 2024-05-08T22:38:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001240020 2024-05-08T22:38:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-05-08T22:38:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-05-08T22:38:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 6, 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.011501900, 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001916983 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-05-08T22:38:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 6, 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.025514012, 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.004252335 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1206, 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 1.158111343, 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000960291 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:38:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 2, 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 1, 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 1, 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 2287, 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 2287, 2024-05-08T22:38:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 193, 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 314805, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 130, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 616, 2024-05-08T22:38:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 7, 2024-05-08T22:38:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-05-08T22:38:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-05-08T22:38:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 7528, 2024-05-08T22:38:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 158, 2024-05-08T22:38:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 69485, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 344, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-05-08T22:38:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 3669, 2024-05-08T22:38:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 12589046, 2024-05-08T22:38:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 27, 2024-05-08T22:38:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 3669, 2024-05-08T22:38:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-05-08T22:38:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3669, 2024-05-08T22:38:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 5.620873710, 2024-05-08T22:38:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001531990 2024-05-08T22:38:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-05-08T22:38:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3669, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 3704 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 3669, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 13, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 3656, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 0, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 10, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 13, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 1247, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 17, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-05-08T22:38:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 2360, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 0, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 20, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-05-08T22:38:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-05-08T22:38:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 81, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 2, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 12, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 12, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 0, 2024-05-08T22:38:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 29, 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 0, 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 8 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 1, 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 0 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 8, 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 5, 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 1212, 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 9870, 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 2495011, 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 9870, 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 9870, 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 2495011, 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 3669, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 12711458, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 3669, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 12711458, 2024-05-08T22:38:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 3669, 2024-05-08T22:38:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 3669, 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 3669, 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 3669, 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 1212, 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 122412, 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 1212, 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 13, 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 122412, 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x560ee2212000": { 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 6070, 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 12359363, 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 6070, 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 1234, 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 12359363, 2024-05-08T22:38:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:29.733 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-05-08T22:38:30.064 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:29 smithi079 ceph-mon[46738]: pgmap v323: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 132 op/s 2024-05-08T22:38:30.066 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-05-08T22:38:30.066 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-05-08T22:38:30.066 DEBUG:tasks.check_counter:Getting stats from l 2024-05-08T22:38:30.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-08T22:38:30.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:29 smithi066 ceph-mon[46830]: pgmap v323: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 132 op/s 2024-05-08T22:38:30.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:29 smithi062 ceph-mon[42571]: pgmap v323: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 132 op/s 2024-05-08T22:38:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T22:38:31.247 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-05-08T22:38:31.247 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T22:38:31.247 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 88, 2024-05-08T22:38:31.247 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 75, 2024-05-08T22:38:31.247 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 494642, 2024-05-08T22:38:31.247 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 37773, 2024-05-08T22:38:31.247 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 5, 2024-05-08T22:38:31.247 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 1, 2024-05-08T22:38:31.247 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.015679208, 2024-05-08T22:38:31.247 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.004106045, 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.012506182, 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.001523135, 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 75, 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.006349254, 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000084656 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.248 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 493808, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 37456 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 135, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 183, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 33146, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 196463, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 5, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.016839383, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.005182384, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.004726417, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.005027978, 2024-05-08T22:38:31.249 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 181, 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.009065512, 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000050085 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 154784 2024-05-08T22:38:31.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 398, 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 400, 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 120171, 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 123836, 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 4, 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 3, 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.039592273, 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.011636190, 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.011193821, 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.012709506, 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T22:38:31.251 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 398, 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.019098295, 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000047985 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-05-08T22:38:31.252 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-05-08T22:38:31.253 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-05-08T22:38:31.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.253 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-05-08T22:38:31.253 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-08T22:38:31.253 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-08T22:38:31.253 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 908, 2024-05-08T22:38:31.253 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.026898171, 2024-05-08T22:38:31.253 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000029623 2024-05-08T22:38:31.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:31.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 0, 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 0, 2024-05-08T22:38:31.254 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-05-08T22:38:31.255 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.255 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.255 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.255 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-05-08T22:38:31.255 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 0, 2024-05-08T22:38:31.255 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-05-08T22:38:31.255 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-05-08T22:38:31.255 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 0, 2024-05-08T22:38:31.255 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 1, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 0, 2024-05-08T22:38:31.256 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-05-08T22:38:31.257 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-05-08T22:38:31.258 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-05-08T22:38:31.258 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 0, 2024-05-08T22:38:31.258 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 0, 2024-05-08T22:38:31.258 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-05-08T22:38:31.258 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T22:38:31.258 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-05-08T22:38:31.258 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T22:38:31.258 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-05-08T22:38:31.258 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-05-08T22:38:31.258 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T22:38:31.258 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T22:38:31.259 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-05-08T22:38:31.259 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-05-08T22:38:31.259 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T22:38:31.259 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 0, 2024-05-08T22:38:31.259 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T22:38:31.259 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T22:38:31.259 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T22:38:31.259 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 0, 2024-05-08T22:38:31.259 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 0, 2024-05-08T22:38:31.259 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 0, 2024-05-08T22:38:31.259 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 0 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 0, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 0, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 0, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 0, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 0, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-05-08T22:38:31.260 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 0, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 0, 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 4, 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 0, 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 1, 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-05-08T22:38:31.262 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-05-08T22:38:31.264 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 4194304, 2024-05-08T22:38:31.264 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 4195692, 2024-05-08T22:38:31.264 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 4195551, 2024-05-08T22:38:31.264 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-05-08T22:38:31.264 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.264 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.264 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 4 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 3, 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 3, 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 0, 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 1, 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 1, 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 0, 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 0, 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 0, 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 0, 2024-05-08T22:38:31.265 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 0, 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 0, 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 0, 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 47336, 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 207156 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 0, 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 0, 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 0, 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 0, 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.266 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-05-08T22:38:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.268 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-05-08T22:38:31.270 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.271 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.271 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.271 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-05-08T22:38:31.271 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.271 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.271 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.272 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-05-08T22:38:31.273 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.275 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.276 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.277 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 2, 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 1, 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 2, 2024-05-08T22:38:31.278 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 0, 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 0, 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 2970, 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-05-08T22:38:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T22:38:31.280 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T22:38:31.281 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T22:38:31.281 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T22:38:31.281 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T22:38:31.281 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-05-08T22:38:31.281 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-05-08T22:38:31.281 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-05-08T22:38:31.281 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T22:38:31.281 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T22:38:31.281 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-05-08T22:38:31.281 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-05-08T22:38:31.281 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-05-08T22:38:31.282 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-05-08T22:38:31.282 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T22:38:31.282 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-05-08T22:38:31.283 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T22:38:31.283 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-05-08T22:38:31.283 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 189279, 2024-05-08T22:38:31.283 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 49, 2024-05-08T22:38:31.283 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-05-08T22:38:31.283 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-05-08T22:38:31.283 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-05-08T22:38:31.283 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-05-08T22:38:31.283 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-05-08T22:38:31.283 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-05-08T22:38:31.283 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 7528, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 158, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 8320, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 13, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-05-08T22:38:31.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 533, 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 21, 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 3, 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 533, 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 533, 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.274993871, 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000515935 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.285 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 533, 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 534 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 533, 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 533, 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 0, 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 328, 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 0, 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 203, 2024-05-08T22:38:31.286 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 1, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-05-08T22:38:31.287 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-05-08T22:38:31.288 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 81, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-05-08T22:38:31.289 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 2, 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 5, 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 5, 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 0, 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 0, 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 2, 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 0 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 0, 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-05-08T22:38:31.290 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 0 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 0, 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 0, 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.291 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 617, 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 585262, 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 617, 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 617, 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 585262, 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:31.292 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 533, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 1393, 2024-05-08T22:38:31.293 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 533, 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 1393, 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:31.294 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 533, 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 533, 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 533, 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 533, 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-05-08T22:38:31.295 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:31.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-05-08T22:38:31.297 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x556fbb5e2140": { 2024-05-08T22:38:31.297 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-05-08T22:38:31.297 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-05-08T22:38:31.297 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-05-08T22:38:31.297 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-05-08T22:38:31.297 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-05-08T22:38:31.297 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-05-08T22:38:31.297 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-05-08T22:38:31.297 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-05-08T22:38:31.297 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-05-08T22:38:31.298 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-05-08T22:38:31.298 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-05-08T22:38:31.298 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-05-08T22:38:31.298 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-05-08T22:38:31.298 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-05-08T22:38:31.298 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-05-08T22:38:31.298 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:31.298 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-05-08T22:38:31.298 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-05-08T22:38:31.638 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-08T22:38:31.638 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-08T22:38:31.638 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T22:38:31.649 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T22:38:31.649 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:38:31.649 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:38:31.674 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-05-08T22:38:31.675 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T22:38:31.736 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:38:31.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.917+0000 7f375cbcb700 1 -- 172.21.15.62:0/1862892371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3758101910 msgr2=0x7f37581081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.917+0000 7f375cbcb700 1 --2- 172.21.15.62:0/1862892371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3758101910 0x7f37581081c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f375000b0d0 tx=0x7f375000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.917+0000 7f375cbcb700 1 -- 172.21.15.62:0/1862892371 shutdown_connections 2024-05-08T22:38:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.917+0000 7f375cbcb700 1 --2- 172.21.15.62:0/1862892371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3758101910 0x7f37581081c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.917+0000 7f375cbcb700 1 --2- 172.21.15.62:0/1862892371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37580ffd80 0x7f37581001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.917+0000 7f375cbcb700 1 --2- 172.21.15.62:0/1862892371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3758100f70 0x7f37581013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.917+0000 7f375cbcb700 1 -- 172.21.15.62:0/1862892371 >> 172.21.15.62:0/1862892371 conn(0x7f37580fb2f0 msgr2=0x7f37580fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.917+0000 7f375cbcb700 1 -- 172.21.15.62:0/1862892371 shutdown_connections 2024-05-08T22:38:31.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.917+0000 7f375cbcb700 1 -- 172.21.15.62:0/1862892371 wait complete. 2024-05-08T22:38:31.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.918+0000 7f375cbcb700 1 Processor -- start 2024-05-08T22:38:31.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.918+0000 7f375cbcb700 1 -- start start 2024-05-08T22:38:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.918+0000 7f375cbcb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37580ffd80 0x7f3758196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.918+0000 7f375cbcb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3758100f70 0x7f37581973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.918+0000 7f375cbcb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3758101910 0x7f375819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.918+0000 7f375cbcb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375819e9c0 con 0x7f3758100f70 2024-05-08T22:38:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.918+0000 7f375cbcb700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375819eb00 con 0x7f37580ffd80 2024-05-08T22:38:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.918+0000 7f375cbcb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375819ec90 con 0x7f3758101910 2024-05-08T22:38:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.918+0000 7f3755d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3758100f70 0x7f37581973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.918+0000 7f3755d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3758100f70 0x7f37581973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58808/0 (socket says 172.21.15.62:58808) 2024-05-08T22:38:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.918+0000 7f3755d9b700 1 -- 172.21.15.62:0/1391722202 learned_addr learned my addr 172.21.15.62:0/1391722202 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f375659c700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37580ffd80 0x7f3758196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f3756d9d700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3758101910 0x7f375819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f3755d9b700 1 -- 172.21.15.62:0/1391722202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3758101910 msgr2=0x7f375819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f3755d9b700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3758101910 0x7f375819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f3755d9b700 1 -- 172.21.15.62:0/1391722202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37580ffd80 msgr2=0x7f3758196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f3755d9b700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37580ffd80 0x7f3758196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f3755d9b700 1 -- 172.21.15.62:0/1391722202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3750009d00 con 0x7f3758100f70 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f3755d9b700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3758100f70 0x7f37581973d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f374800c9b0 tx=0x7f374800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f37477fe700 1 -- 172.21.15.62:0/1391722202 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748007b00 con 0x7f3758100f70 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f37477fe700 1 -- 172.21.15.62:0/1391722202 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f374800ce50 con 0x7f3758100f70 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f375cbcb700 1 -- 172.21.15.62:0/1391722202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375819ef70 con 0x7f3758100f70 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f37477fe700 1 -- 172.21.15.62:0/1391722202 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748018660 con 0x7f3758100f70 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.919+0000 7f375cbcb700 1 -- 172.21.15.62:0/1391722202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375819f470 con 0x7f3758100f70 2024-05-08T22:38:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.920+0000 7f375cbcb700 1 -- 172.21.15.62:0/1391722202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3758067480 con 0x7f3758100f70 2024-05-08T22:38:31.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.922+0000 7f37477fe700 1 -- 172.21.15.62:0/1391722202 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f374800f450 con 0x7f3758100f70 2024-05-08T22:38:31.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.923+0000 7f37477fe700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f37400a1120 0x7f37400a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.923+0000 7f375659c700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f37400a1120 0x7f37400a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.923+0000 7f37477fe700 1 -- 172.21.15.62:0/1391722202 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f37480cca40 con 0x7f3758100f70 2024-05-08T22:38:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.923+0000 7f375659c700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f37400a1120 0x7f37400a35e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3758100dd0 tx=0x7f374c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:31.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:31.926+0000 7f37477fe700 1 -- 172.21.15.62:0/1391722202 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3748096270 con 0x7f3758100f70 2024-05-08T22:38:32.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:31 smithi066 ceph-mon[46830]: pgmap v324: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 70 op/s 2024-05-08T22:38:32.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.138+0000 7f375cbcb700 1 -- 172.21.15.62:0/1391722202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f375810b630 con 0x7f3758100f70 2024-05-08T22:38:32.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:31 smithi079 ceph-mon[46738]: pgmap v324: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 70 op/s 2024-05-08T22:38:32.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.139+0000 7f37477fe700 1 -- 172.21.15.62:0/1391722202 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22481 (secure 0 0 0) 0x7f3748007e60 con 0x7f3758100f70 2024-05-08T22:38:32.139 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:38:32.139 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:36:05.960694+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 -- 172.21.15.62:0/1391722202 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f37400a1120 msgr2=0x7f37400a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f37400a1120 0x7f37400a35e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3758100dd0 tx=0x7f374c009190 comp rx=0 tx=0).stop 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 -- 172.21.15.62:0/1391722202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3758100f70 msgr2=0x7f37581973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3758100f70 0x7f37581973d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f374800c9b0 tx=0x7f374800ccc0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 -- 172.21.15.62:0/1391722202 shutdown_connections 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3758101910 0x7f375819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f37400a1120 0x7f37400a35e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37580ffd80 0x7f3758196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 --2- 172.21.15.62:0/1391722202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3758100f70 0x7f37581973d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 -- 172.21.15.62:0/1391722202 >> 172.21.15.62:0/1391722202 conn(0x7f37580fb2f0 msgr2=0x7f37580fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 -- 172.21.15.62:0/1391722202 shutdown_connections 2024-05-08T22:38:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.142+0000 7f375cbcb700 1 -- 172.21.15.62:0/1391722202 wait complete. 2024-05-08T22:38:32.152 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:31 smithi062 ceph-mon[42571]: pgmap v324: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 70 op/s 2024-05-08T22:38:32.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T22:31:54.466249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T22:35:08.296334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T22:35:09.167901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T22:38:32.154 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T22:38:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.286+0000 7f90b2fa1700 1 -- 172.21.15.62:0/938229297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac104070 msgr2=0x7f90ac106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.286+0000 7f90b2fa1700 1 --2- 172.21.15.62:0/938229297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac104070 0x7f90ac106460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f909c009990 tx=0x7f909c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.286+0000 7f90b2fa1700 1 -- 172.21.15.62:0/938229297 shutdown_connections 2024-05-08T22:38:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.286+0000 7f90b2fa1700 1 --2- 172.21.15.62:0/938229297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90ac1069a0 0x7f90ac108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.286+0000 7f90b2fa1700 1 --2- 172.21.15.62:0/938229297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac101740 0x7f90ac103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.286+0000 7f90b2fa1700 1 --2- 172.21.15.62:0/938229297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac104070 0x7f90ac106460 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.286+0000 7f90b2fa1700 1 -- 172.21.15.62:0/938229297 >> 172.21.15.62:0/938229297 conn(0x7f90ac0fb130 msgr2=0x7f90ac0fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.286+0000 7f90b2fa1700 1 -- 172.21.15.62:0/938229297 shutdown_connections 2024-05-08T22:38:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.286+0000 7f90b2fa1700 1 -- 172.21.15.62:0/938229297 wait complete. 2024-05-08T22:38:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b2fa1700 1 Processor -- start 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b2fa1700 1 -- start start 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b2fa1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90ac101740 0x7f90ac196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b2fa1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac104070 0x7f90ac197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b2fa1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac1069a0 0x7f90ac19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b2fa1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac19e640 con 0x7f90ac1069a0 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b2fa1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac19e780 con 0x7f90ac101740 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b2fa1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac19e910 con 0x7f90ac104070 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b27a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac1069a0 0x7f90ac19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b27a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac1069a0 0x7f90ac19e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58814/0 (socket says 172.21.15.62:58814) 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b27a0700 1 -- 172.21.15.62:0/970784213 learned_addr learned my addr 172.21.15.62:0/970784213 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b179e700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac104070 0x7f90ac197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b1f9f700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90ac101740 0x7f90ac196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b27a0700 1 -- 172.21.15.62:0/970784213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac104070 msgr2=0x7f90ac197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b27a0700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac104070 0x7f90ac197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b27a0700 1 -- 172.21.15.62:0/970784213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90ac101740 msgr2=0x7f90ac196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b27a0700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90ac101740 0x7f90ac196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b27a0700 1 -- 172.21.15.62:0/970784213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0095f0 con 0x7f90ac1069a0 2024-05-08T22:38:32.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b1f9f700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90ac101740 0x7f90ac196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:38:32.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.287+0000 7f90b27a0700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac1069a0 0x7f90ac19e070 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f90a400ef30 tx=0x7f90a400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:32.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.288+0000 7f90a2ffd700 1 -- 172.21.15.62:0/970784213 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4018070 con 0x7f90ac1069a0 2024-05-08T22:38:32.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.288+0000 7f90b2fa1700 1 -- 172.21.15.62:0/970784213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ac19eb90 con 0x7f90ac1069a0 2024-05-08T22:38:32.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.288+0000 7f90a2ffd700 1 -- 172.21.15.62:0/970784213 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90a400f040 con 0x7f90ac1069a0 2024-05-08T22:38:32.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.288+0000 7f90a2ffd700 1 -- 172.21.15.62:0/970784213 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4003dc0 con 0x7f90ac1069a0 2024-05-08T22:38:32.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.288+0000 7f90b2fa1700 1 -- 172.21.15.62:0/970784213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ac19f0a0 con 0x7f90ac1069a0 2024-05-08T22:38:32.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.289+0000 7f90b2fa1700 1 -- 172.21.15.62:0/970784213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ac0fccb0 con 0x7f90ac1069a0 2024-05-08T22:38:32.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.290+0000 7f90a2ffd700 1 -- 172.21.15.62:0/970784213 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f90a400caa0 con 0x7f90ac1069a0 2024-05-08T22:38:32.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.290+0000 7f90a2ffd700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f90980a11c0 0x7f90980a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.290+0000 7f90a2ffd700 1 -- 172.21.15.62:0/970784213 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f90a40ccf40 con 0x7f90ac1069a0 2024-05-08T22:38:32.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.291+0000 7f90b1f9f700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f90980a11c0 0x7f90980a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.291+0000 7f90b1f9f700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f90980a11c0 0x7f90980a3680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009be0 tx=0x7f90a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:32.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.294+0000 7f90a2ffd700 1 -- 172.21.15.62:0/970784213 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f90a4096770 con 0x7f90ac1069a0 2024-05-08T22:38:32.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.501+0000 7f90b2fa1700 1 -- 172.21.15.62:0/970784213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90ac04ec60 con 0x7f90ac1069a0 2024-05-08T22:38:32.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.502+0000 7f90a2ffd700 1 -- 172.21.15.62:0/970784213 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90a4099d90 con 0x7f90ac1069a0 2024-05-08T22:38:32.501 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-05-08T22:38:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 -- 172.21.15.62:0/970784213 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f90980a11c0 msgr2=0x7f90980a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f90980a11c0 0x7f90980a3680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009be0 tx=0x7f90a8009260 comp rx=0 tx=0).stop 2024-05-08T22:38:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 -- 172.21.15.62:0/970784213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac1069a0 msgr2=0x7f90ac19e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac1069a0 0x7f90ac19e070 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f90a400ef30 tx=0x7f90a400c450 comp rx=0 tx=0).stop 2024-05-08T22:38:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 -- 172.21.15.62:0/970784213 shutdown_connections 2024-05-08T22:38:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac1069a0 0x7f90ac19e070 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f90980a11c0 0x7f90980a3680 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90ac101740 0x7f90ac196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 --2- 172.21.15.62:0/970784213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac104070 0x7f90ac197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 -- 172.21.15.62:0/970784213 >> 172.21.15.62:0/970784213 conn(0x7f90ac0fb130 msgr2=0x7f90ac1021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 -- 172.21.15.62:0/970784213 shutdown_connections 2024-05-08T22:38:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.504+0000 7f90b2fa1700 1 -- 172.21.15.62:0/970784213 wait complete. 2024-05-08T22:38:32.516 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T22:38:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.647+0000 7f42713b0700 1 -- 172.21.15.62:0/1887620950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c101320 msgr2=0x7f426c101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.647+0000 7f42713b0700 1 --2- 172.21.15.62:0/1887620950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c101320 0x7f426c101740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f425c009990 tx=0x7f425c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.647+0000 7f42713b0700 1 -- 172.21.15.62:0/1887620950 shutdown_connections 2024-05-08T22:38:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.647+0000 7f42713b0700 1 --2- 172.21.15.62:0/1887620950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f426c1009f0 0x7f426c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.647+0000 7f42713b0700 1 --2- 172.21.15.62:0/1887620950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c101320 0x7f426c101740 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.647+0000 7f42713b0700 1 --2- 172.21.15.62:0/1887620950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f426c100050 0x7f426c1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.647+0000 7f42713b0700 1 -- 172.21.15.62:0/1887620950 >> 172.21.15.62:0/1887620950 conn(0x7f426c0faa60 msgr2=0x7f426c0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:32.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.647+0000 7f42713b0700 1 -- 172.21.15.62:0/1887620950 shutdown_connections 2024-05-08T22:38:32.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.647+0000 7f42713b0700 1 -- 172.21.15.62:0/1887620950 wait complete. 2024-05-08T22:38:32.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.648+0000 7f42713b0700 1 Processor -- start 2024-05-08T22:38:32.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.648+0000 7f42713b0700 1 -- start start 2024-05-08T22:38:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f42713b0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f426c100050 0x7f426c195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f42713b0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1009f0 0x7f426c196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f42713b0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f426c101320 0x7f426c19d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f42713b0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c19d7f0 con 0x7f426c1009f0 2024-05-08T22:38:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f42713b0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c19d960 con 0x7f426c100050 2024-05-08T22:38:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f42713b0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c19db20 con 0x7f426c101320 2024-05-08T22:38:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f426a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1009f0 0x7f426c196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f426a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1009f0 0x7f426c196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58826/0 (socket says 172.21.15.62:58826) 2024-05-08T22:38:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f426a7fc700 1 -- 172.21.15.62:0/1659499940 learned_addr learned my addr 172.21.15.62:0/1659499940 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f426affd700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f426c100050 0x7f426c195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f426a7fc700 1 -- 172.21.15.62:0/1659499940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f426c101320 msgr2=0x7f426c19d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f426a7fc700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f426c101320 0x7f426c19d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f426a7fc700 1 -- 172.21.15.62:0/1659499940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f426c100050 msgr2=0x7f426c195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f426a7fc700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f426c100050 0x7f426c195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f426a7fc700 1 -- 172.21.15.62:0/1659499940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0095f0 con 0x7f426c1009f0 2024-05-08T22:38:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f426a7fc700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1009f0 0x7f426c196210 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4260009970 tx=0x7f4260009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f4253fff700 1 -- 172.21.15.62:0/1659499940 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4260012070 con 0x7f426c1009f0 2024-05-08T22:38:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f42713b0700 1 -- 172.21.15.62:0/1659499940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c0fefa0 con 0x7f426c1009f0 2024-05-08T22:38:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f4253fff700 1 -- 172.21.15.62:0/1659499940 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f426000d260 con 0x7f426c1009f0 2024-05-08T22:38:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f4253fff700 1 -- 172.21.15.62:0/1659499940 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600179d0 con 0x7f426c1009f0 2024-05-08T22:38:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.649+0000 7f42713b0700 1 -- 172.21.15.62:0/1659499940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c0ff4b0 con 0x7f426c1009f0 2024-05-08T22:38:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.650+0000 7f42713b0700 1 -- 172.21.15.62:0/1659499940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426c068160 con 0x7f426c1009f0 2024-05-08T22:38:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.652+0000 7f4253fff700 1 -- 172.21.15.62:0/1659499940 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4260007c00 con 0x7f426c1009f0 2024-05-08T22:38:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.652+0000 7f4253fff700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42540a1120 0x7f42540a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.652+0000 7f426affd700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42540a1120 0x7f42540a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.652+0000 7f4253fff700 1 -- 172.21.15.62:0/1659499940 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f42600cdc10 con 0x7f426c1009f0 2024-05-08T22:38:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.653+0000 7f426affd700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42540a1120 0x7f42540a35e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f426c0fc6a0 tx=0x7f425c003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:32.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.656+0000 7f4253fff700 1 -- 172.21.15.62:0/1659499940 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f42600973b0 con 0x7f426c1009f0 2024-05-08T22:38:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.865+0000 7f42713b0700 1 -- 172.21.15.62:0/1659499940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f426c10b710 con 0x7f426c1009f0 2024-05-08T22:38:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.866+0000 7f4253fff700 1 -- 172.21.15.62:0/1659499940 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f426c10b710 con 0x7f426c1009f0 2024-05-08T22:38:32.865 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-05-08T22:38:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 -- 172.21.15.62:0/1659499940 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42540a1120 msgr2=0x7f42540a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42540a1120 0x7f42540a35e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f426c0fc6a0 tx=0x7f425c003790 comp rx=0 tx=0).stop 2024-05-08T22:38:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 -- 172.21.15.62:0/1659499940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1009f0 msgr2=0x7f426c196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1009f0 0x7f426c196210 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4260009970 tx=0x7f4260009c80 comp rx=0 tx=0).stop 2024-05-08T22:38:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 -- 172.21.15.62:0/1659499940 shutdown_connections 2024-05-08T22:38:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f426c101320 0x7f426c19d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f42540a1120 0x7f42540a35e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f426c100050 0x7f426c195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 --2- 172.21.15.62:0/1659499940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1009f0 0x7f426c196210 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 -- 172.21.15.62:0/1659499940 >> 172.21.15.62:0/1659499940 conn(0x7f426c0faa60 msgr2=0x7f426c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:32.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 -- 172.21.15.62:0/1659499940 shutdown_connections 2024-05-08T22:38:32.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:32.868+0000 7f42713b0700 1 -- 172.21.15.62:0/1659499940 wait complete. 2024-05-08T22:38:32.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:32 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1391722202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:38:32.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:32 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/970784213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:38:32.880 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T22:38:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 -- 172.21.15.62:0/3512693940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd81018f0 msgr2=0x7f3bd81081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 --2- 172.21.15.62:0/3512693940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd81018f0 0x7f3bd81081a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000b0d0 tx=0x7f3bd000b3e0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 -- 172.21.15.62:0/3512693940 shutdown_connections 2024-05-08T22:38:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 --2- 172.21.15.62:0/3512693940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd81018f0 0x7f3bd81081a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 --2- 172.21.15.62:0/3512693940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bd80ffd60 0x7f3bd8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 --2- 172.21.15.62:0/3512693940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bd8100f50 0x7f3bd81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 -- 172.21.15.62:0/3512693940 >> 172.21.15.62:0/3512693940 conn(0x7f3bd80fb290 msgr2=0x7f3bd80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 -- 172.21.15.62:0/3512693940 shutdown_connections 2024-05-08T22:38:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 -- 172.21.15.62:0/3512693940 wait complete. 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 Processor -- start 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 -- start start 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bd80ffd60 0x7f3bd8196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd8100f50 0x7f3bd81974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bd81018f0 0x7f3bd819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd819ea70 con 0x7f3bd8100f50 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd819ebb0 con 0x7f3bd81018f0 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.011+0000 7f3bdead9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd819ed40 con 0x7f3bd80ffd60 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd8100f50 0x7f3bd81974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd8100f50 0x7f3bd81974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58846/0 (socket says 172.21.15.62:58846) 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd7fff700 1 -- 172.21.15.62:0/2756981238 learned_addr learned my addr 172.21.15.62:0/2756981238 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bdd076700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bd81018f0 0x7f3bd819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bdc875700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bd80ffd60 0x7f3bd8196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:33.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd7fff700 1 -- 172.21.15.62:0/2756981238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bd80ffd60 msgr2=0x7f3bd8196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:33.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd7fff700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bd80ffd60 0x7f3bd8196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd7fff700 1 -- 172.21.15.62:0/2756981238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bd81018f0 msgr2=0x7f3bd819e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:33.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd7fff700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bd81018f0 0x7f3bd819e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd7fff700 1 -- 172.21.15.62:0/2756981238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd0009d00 con 0x7f3bd8100f50 2024-05-08T22:38:33.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd7fff700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd8100f50 0x7f3bd81974d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b530 tx=0x7f3bc400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:33.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd5ffb700 1 -- 172.21.15.62:0/2756981238 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4011830 con 0x7f3bd8100f50 2024-05-08T22:38:33.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd5ffb700 1 -- 172.21.15.62:0/2756981238 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bc4011e50 con 0x7f3bd8100f50 2024-05-08T22:38:33.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bdead9700 1 -- 172.21.15.62:0/2756981238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd819efc0 con 0x7f3bd8100f50 2024-05-08T22:38:33.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bd5ffb700 1 -- 172.21.15.62:0/2756981238 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc400f2e0 con 0x7f3bd8100f50 2024-05-08T22:38:33.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.012+0000 7f3bdead9700 1 -- 172.21.15.62:0/2756981238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd819f570 con 0x7f3bd8100f50 2024-05-08T22:38:33.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.013+0000 7f3bdead9700 1 -- 172.21.15.62:0/2756981238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd8107110 con 0x7f3bd8100f50 2024-05-08T22:38:33.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.014+0000 7f3bd5ffb700 1 -- 172.21.15.62:0/2756981238 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3bc4010380 con 0x7f3bd8100f50 2024-05-08T22:38:33.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.015+0000 7f3bd5ffb700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3bc80a11a0 0x7f3bc80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:33.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.015+0000 7f3bd5ffb700 1 -- 172.21.15.62:0/2756981238 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3bc40cc3d0 con 0x7f3bd8100f50 2024-05-08T22:38:33.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.015+0000 7f3bdc875700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3bc80a11a0 0x7f3bc80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:33.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.016+0000 7f3bdc875700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3bc80a11a0 0x7f3bc80a3660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc005fd0 tx=0x7f3bcc005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.020+0000 7f3bd5ffb700 1 -- 172.21.15.62:0/2756981238 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3bc4095b70 con 0x7f3bd8100f50 2024-05-08T22:38:33.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:32 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1391722202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:38:33.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:32 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/970784213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:38:33.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:32 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1391722202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:38:33.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:32 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/970784213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T22:38:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.230+0000 7f3bdead9700 1 -- 172.21.15.62:0/2756981238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3bd802ce40 con 0x7f3bd8100f50 2024-05-08T22:38:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.230+0000 7f3bd5ffb700 1 -- 172.21.15.62:0/2756981238 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3bc4099190 con 0x7f3bd8100f50 2024-05-08T22:38:33.229 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-05-08T22:38:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.232+0000 7f3bdead9700 1 -- 172.21.15.62:0/2756981238 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3bc80a11a0 msgr2=0x7f3bc80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.232+0000 7f3bdead9700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3bc80a11a0 0x7f3bc80a3660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc005fd0 tx=0x7f3bcc005dd0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.232+0000 7f3bdead9700 1 -- 172.21.15.62:0/2756981238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd8100f50 msgr2=0x7f3bd81974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.232+0000 7f3bdead9700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd8100f50 0x7f3bd81974d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b530 tx=0x7f3bc400b840 comp rx=0 tx=0).stop 2024-05-08T22:38:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.232+0000 7f3bdead9700 1 -- 172.21.15.62:0/2756981238 shutdown_connections 2024-05-08T22:38:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.232+0000 7f3bdead9700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f3bc80a11a0 0x7f3bc80a3660 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.232+0000 7f3bdead9700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bd81018f0 0x7f3bd819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.233+0000 7f3bdead9700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bd80ffd60 0x7f3bd8196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.233+0000 7f3bdead9700 1 --2- 172.21.15.62:0/2756981238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd8100f50 0x7f3bd81974d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.233+0000 7f3bdead9700 1 -- 172.21.15.62:0/2756981238 >> 172.21.15.62:0/2756981238 conn(0x7f3bd80fb290 msgr2=0x7f3bd8104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.233+0000 7f3bdead9700 1 -- 172.21.15.62:0/2756981238 shutdown_connections 2024-05-08T22:38:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.233+0000 7f3bdead9700 1 -- 172.21.15.62:0/2756981238 wait complete. 2024-05-08T22:38:33.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T22:38:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.373+0000 7f361d2c9700 1 -- 172.21.15.62:0/449048636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3618104220 msgr2=0x7f3618106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.373+0000 7f361d2c9700 1 --2- 172.21.15.62:0/449048636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3618104220 0x7f3618106610 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f360c009990 tx=0x7f360c009ca0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.373+0000 7f361d2c9700 1 -- 172.21.15.62:0/449048636 shutdown_connections 2024-05-08T22:38:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.373+0000 7f361d2c9700 1 --2- 172.21.15.62:0/449048636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3618106b50 0x7f3618108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.373+0000 7f361d2c9700 1 --2- 172.21.15.62:0/449048636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36181018f0 0x7f3618103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.373+0000 7f361d2c9700 1 --2- 172.21.15.62:0/449048636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3618104220 0x7f3618106610 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.373+0000 7f361d2c9700 1 -- 172.21.15.62:0/449048636 >> 172.21.15.62:0/449048636 conn(0x7f36180fb300 msgr2=0x7f36180fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.373+0000 7f361d2c9700 1 -- 172.21.15.62:0/449048636 shutdown_connections 2024-05-08T22:38:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.373+0000 7f361d2c9700 1 -- 172.21.15.62:0/449048636 wait complete. 2024-05-08T22:38:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.373+0000 7f361d2c9700 1 Processor -- start 2024-05-08T22:38:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.373+0000 7f361d2c9700 1 -- start start 2024-05-08T22:38:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f361d2c9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36181018f0 0x7f3618196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f361d2c9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3618104220 0x7f3618197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f361d2c9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3618106b50 0x7f361819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f361d2c9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361819e830 con 0x7f3618104220 2024-05-08T22:38:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f361d2c9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361819e9a0 con 0x7f3618106b50 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f361d2c9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361819eb60 con 0x7f36181018f0 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f36167fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3618104220 0x7f3618197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f36167fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3618104220 0x7f3618197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58860/0 (socket says 172.21.15.62:58860) 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f36167fc700 1 -- 172.21.15.62:0/994609229 learned_addr learned my addr 172.21.15.62:0/994609229 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f3616ffd700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36181018f0 0x7f3618196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f36167fc700 1 -- 172.21.15.62:0/994609229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36181018f0 msgr2=0x7f3618196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f36177fe700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3618106b50 0x7f361819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f36167fc700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36181018f0 0x7f3618196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f36167fc700 1 -- 172.21.15.62:0/994609229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3618106b50 msgr2=0x7f361819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f36167fc700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3618106b50 0x7f361819e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f36167fc700 1 -- 172.21.15.62:0/994609229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c0095f0 con 0x7f3618104220 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f3616ffd700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36181018f0 0x7f3618196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f36167fc700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3618104220 0x7f3618197280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f360c0095c0 tx=0x7f360c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f35fffff700 1 -- 172.21.15.62:0/994609229 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c01d070 con 0x7f3618104220 2024-05-08T22:38:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f35fffff700 1 -- 172.21.15.62:0/994609229 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f360c005510 con 0x7f3618104220 2024-05-08T22:38:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f361d2c9700 1 -- 172.21.15.62:0/994609229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361819ede0 con 0x7f3618104220 2024-05-08T22:38:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.374+0000 7f35fffff700 1 -- 172.21.15.62:0/994609229 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c004cd0 con 0x7f3618104220 2024-05-08T22:38:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.375+0000 7f361d2c9700 1 -- 172.21.15.62:0/994609229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361819f1b0 con 0x7f3618104220 2024-05-08T22:38:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.375+0000 7f361d2c9700 1 -- 172.21.15.62:0/994609229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36180fce80 con 0x7f3618104220 2024-05-08T22:38:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.376+0000 7f35fffff700 1 -- 172.21.15.62:0/994609229 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f360c00fc40 con 0x7f3618104220 2024-05-08T22:38:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.377+0000 7f35fffff700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f36000a11a0 0x7f36000a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.377+0000 7f3616ffd700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f36000a11a0 0x7f36000a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.377+0000 7f35fffff700 1 -- 172.21.15.62:0/994609229 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f360c0cd860 con 0x7f3618104220 2024-05-08T22:38:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.378+0000 7f3616ffd700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f36000a11a0 0x7f36000a3660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3608005fd0 tx=0x7f3608005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.381+0000 7f35fffff700 1 -- 172.21.15.62:0/994609229 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f360c097090 con 0x7f3618104220 2024-05-08T22:38:33.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.605+0000 7f361d2c9700 1 -- 172.21.15.62:0/994609229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3618068160 con 0x7f3618104220 2024-05-08T22:38:33.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.606+0000 7f35fffff700 1 -- 172.21.15.62:0/994609229 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) v1 ==== 81+0+590 (secure 0 0 0) 0x7f360c09a6b0 con 0x7f3618104220 2024-05-08T22:38:33.605 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/766518770 2024-05-09T22:30:03.951072+0000 2024-05-08T22:38:33.605 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/766518770 2024-05-09T22:30:03.951072+0000 2024-05-08T22:38:33.605 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2477775765 2024-05-09T22:30:03.951072+0000 2024-05-08T22:38:33.605 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2433771662 2024-05-09T22:30:03.951072+0000 2024-05-08T22:38:33.605 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1318760241 2024-05-09T22:29:43.058069+0000 2024-05-08T22:38:33.605 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1247031049 2024-05-09T22:29:43.058069+0000 2024-05-08T22:38:33.605 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/3409257313 2024-05-09T22:29:43.058069+0000 2024-05-08T22:38:33.605 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/3409257313 2024-05-09T22:29:43.058069+0000 2024-05-08T22:38:33.605 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3020122046 2024-05-09T22:29:43.058069+0000 2024-05-08T22:38:33.606 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2719135336 2024-05-09T22:30:03.951072+0000 2024-05-08T22:38:33.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 -- 172.21.15.62:0/994609229 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f36000a11a0 msgr2=0x7f36000a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:33.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f36000a11a0 0x7f36000a3660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3608005fd0 tx=0x7f3608005f00 comp rx=0 tx=0).stop 2024-05-08T22:38:33.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 -- 172.21.15.62:0/994609229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3618104220 msgr2=0x7f3618197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:33.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3618104220 0x7f3618197280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f360c0095c0 tx=0x7f360c00f690 comp rx=0 tx=0).stop 2024-05-08T22:38:33.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 -- 172.21.15.62:0/994609229 shutdown_connections 2024-05-08T22:38:33.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f36000a11a0 0x7f36000a3660 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3618106b50 0x7f361819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36181018f0 0x7f3618196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 --2- 172.21.15.62:0/994609229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3618104220 0x7f3618197280 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:33.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 -- 172.21.15.62:0/994609229 >> 172.21.15.62:0/994609229 conn(0x7f36180fb300 msgr2=0x7f3618102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:33.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 -- 172.21.15.62:0/994609229 shutdown_connections 2024-05-08T22:38:33.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:33.608+0000 7f361d2c9700 1 -- 172.21.15.62:0/994609229 wait complete. 2024-05-08T22:38:33.608 INFO:teuthology.orchestra.run.smithi062.stderr:listed 10 entries 2024-05-08T22:38:33.619 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T22:38:33.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:33.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T22:38:33.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-05-08T22:38:33.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:33.677 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T22:38:33.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:33.718 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-05-08T22:38:33.775 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T22:38:33.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:33.776 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T22:38:33.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T22:38:33.850 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-05-08T22:38:33.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:33 smithi062 ceph-mon[42571]: pgmap v325: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 12 op/s 2024-05-08T22:38:33.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:33 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1659499940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:38:33.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:33 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2756981238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:38:33.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:33 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/994609229' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:38:33.941 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-08T22:38:33.952 INFO:tasks.cephadm:Teardown begin 2024-05-08T22:38:33.952 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T22:38:33.953 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:38:34.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:33 smithi066 ceph-mon[46830]: pgmap v325: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 12 op/s 2024-05-08T22:38:34.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:33 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1659499940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:38:34.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:33 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2756981238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:38:34.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:33 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/994609229' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:38:34.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.083+0000 7fb7d33b0700 1 -- 172.21.15.62:0/702017061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7cc104230 msgr2=0x7fb7cc106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.083+0000 7fb7d33b0700 1 --2- 172.21.15.62:0/702017061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7cc104230 0x7fb7cc106620 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009990 tx=0x7fb7c8009ca0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.083+0000 7fb7d33b0700 1 -- 172.21.15.62:0/702017061 shutdown_connections 2024-05-08T22:38:34.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.083+0000 7fb7d33b0700 1 --2- 172.21.15.62:0/702017061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc106b60 0x7fb7cc108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.083+0000 7fb7d33b0700 1 --2- 172.21.15.62:0/702017061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc101900 0x7fb7cc103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.083+0000 7fb7d33b0700 1 --2- 172.21.15.62:0/702017061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7cc104230 0x7fb7cc106620 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.083+0000 7fb7d33b0700 1 -- 172.21.15.62:0/702017061 >> 172.21.15.62:0/702017061 conn(0x7fb7cc0fb2f0 msgr2=0x7fb7cc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:34.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.083+0000 7fb7d33b0700 1 -- 172.21.15.62:0/702017061 shutdown_connections 2024-05-08T22:38:34.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.083+0000 7fb7d33b0700 1 -- 172.21.15.62:0/702017061 wait complete. 2024-05-08T22:38:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d33b0700 1 Processor -- start 2024-05-08T22:38:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d33b0700 1 -- start start 2024-05-08T22:38:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d33b0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc101900 0x7fb7cc101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d33b0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc104230 0x7fb7cc0ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d33b0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7cc106b60 0x7fb7cc0ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d33b0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc100590 con 0x7fb7cc106b60 2024-05-08T22:38:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d33b0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc100700 con 0x7fb7cc104230 2024-05-08T22:38:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d33b0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc1a0e30 con 0x7fb7cc101900 2024-05-08T22:38:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d194d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7cc106b60 0x7fb7cc0ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d194d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7cc106b60 0x7fb7cc0ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58882/0 (socket says 172.21.15.62:58882) 2024-05-08T22:38:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d194d700 1 -- 172.21.15.62:0/63744544 learned_addr learned my addr 172.21.15.62:0/63744544 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d114c700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc101900 0x7fb7cc101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d094b700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc104230 0x7fb7cc0ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d194d700 1 -- 172.21.15.62:0/63744544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc101900 msgr2=0x7fb7cc101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d194d700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc101900 0x7fb7cc101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d194d700 1 -- 172.21.15.62:0/63744544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc104230 msgr2=0x7fb7cc0ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d194d700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc104230 0x7fb7cc0ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.084+0000 7fb7d194d700 1 -- 172.21.15.62:0/63744544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c80095f0 con 0x7fb7cc106b60 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.085+0000 7fb7d194d700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7cc106b60 0x7fb7cc0ffdf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4007ee0 tx=0x7fb7c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.085+0000 7fb7c27fc700 1 -- 172.21.15.62:0/63744544 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4017070 con 0x7fb7cc106b60 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.085+0000 7fb7c27fc700 1 -- 172.21.15.62:0/63744544 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c400f040 con 0x7fb7cc106b60 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.085+0000 7fb7c27fc700 1 -- 172.21.15.62:0/63744544 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c40054b0 con 0x7fb7cc106b60 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.085+0000 7fb7d33b0700 1 -- 172.21.15.62:0/63744544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7cc1a10c0 con 0x7fb7cc106b60 2024-05-08T22:38:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.085+0000 7fb7d33b0700 1 -- 172.21.15.62:0/63744544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc1a1580 con 0x7fb7cc106b60 2024-05-08T22:38:34.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.086+0000 7fb7d33b0700 1 -- 172.21.15.62:0/63744544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7cc0fce70 con 0x7fb7cc106b60 2024-05-08T22:38:34.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.087+0000 7fb7c27fc700 1 -- 172.21.15.62:0/63744544 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb7c4004ad0 con 0x7fb7cc106b60 2024-05-08T22:38:34.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.087+0000 7fb7c27fc700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7b80a11a0 0x7fb7b80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.087+0000 7fb7c27fc700 1 -- 172.21.15.62:0/63744544 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb7c40cc630 con 0x7fb7cc106b60 2024-05-08T22:38:34.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.087+0000 7fb7d114c700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7b80a11a0 0x7fb7b80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.088+0000 7fb7d114c700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7b80a11a0 0x7fb7b80a3660 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005950 tx=0x7fb7bc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:34.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.091+0000 7fb7c27fc700 1 -- 172.21.15.62:0/63744544 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb7c4095dd0 con 0x7fb7cc106b60 2024-05-08T22:38:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:33 smithi079 ceph-mon[46738]: pgmap v325: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 12 op/s 2024-05-08T22:38:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:33 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1659499940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T22:38:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:33 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2756981238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T22:38:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:33 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/994609229' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T22:38:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.349+0000 7fb7d33b0700 1 -- 172.21.15.62:0/63744544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7cc10c670 con 0x7fb7cc106b60 2024-05-08T22:38:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.350+0000 7fb7c27fc700 1 -- 172.21.15.62:0/63744544 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10899 (secure 0 0 0) 0x7fb7c40993f0 con 0x7fb7cc106b60 2024-05-08T22:38:34.348 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:38:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:35:09.803459+0000","modified":"2024-05-08T22:38:24.738277+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24529,"mds_2":24505,"mds_3":24503,"mds_4":14661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/1756743060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1756743060},{"type":"v1","addr":"172.21.15.62:6839","nonce":1756743060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14661":{"gid":14661,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/780017086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":780017086},{"type":"v1","addr":"172.21.15.62:6841","nonce":780017086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.79:6839/3534785027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":3534785027},{"type":"v1","addr":"172.21.15.79:6839","nonce":3534785027}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/3474672912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":3474672912},{"type":"v1","addr":"172.21.15.66:6839","nonce":3474672912}]},"join_fscid":-1,"export_targets":[4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:38:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 -- 172.21.15.62:0/63744544 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7b80a11a0 msgr2=0x7fb7b80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7b80a11a0 0x7fb7b80a3660 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005950 tx=0x7fb7bc00b410 comp rx=0 tx=0).stop 2024-05-08T22:38:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 -- 172.21.15.62:0/63744544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7cc106b60 msgr2=0x7fb7cc0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7cc106b60 0x7fb7cc0ffdf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4007ee0 tx=0x7fb7c400d3b0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 -- 172.21.15.62:0/63744544 shutdown_connections 2024-05-08T22:38:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7cc106b60 0x7fb7cc0ffdf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc101900 0x7fb7cc101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7fb7b80a11a0 0x7fb7b80a3660 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 --2- 172.21.15.62:0/63744544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7cc104230 0x7fb7cc0ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 -- 172.21.15.62:0/63744544 >> 172.21.15.62:0/63744544 conn(0x7fb7cc0fb2f0 msgr2=0x7fb7cc1037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:34.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 -- 172.21.15.62:0/63744544 shutdown_connections 2024-05-08T22:38:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.352+0000 7fb7d33b0700 1 -- 172.21.15.62:0/63744544 wait complete. 2024-05-08T22:38:34.352 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 31 2024-05-08T22:38:34.363 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T22:38:34.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.491+0000 7f958a5fa700 1 -- 172.21.15.62:0/3703096469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95840ffd80 msgr2=0x7f95841001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.491+0000 7f958a5fa700 1 --2- 172.21.15.62:0/3703096469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95840ffd80 0x7f95841001a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f95740099c0 tx=0x7f9574009cd0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.491+0000 7f958a5fa700 1 -- 172.21.15.62:0/3703096469 shutdown_connections 2024-05-08T22:38:34.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.491+0000 7f958a5fa700 1 --2- 172.21.15.62:0/3703096469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9584101910 0x7f95841081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.491+0000 7f958a5fa700 1 --2- 172.21.15.62:0/3703096469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95840ffd80 0x7f95841001a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.491+0000 7f958a5fa700 1 --2- 172.21.15.62:0/3703096469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9584100f70 0x7f95841013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.491+0000 7f958a5fa700 1 -- 172.21.15.62:0/3703096469 >> 172.21.15.62:0/3703096469 conn(0x7f95840fb310 msgr2=0x7f95840fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.491+0000 7f958a5fa700 1 -- 172.21.15.62:0/3703096469 shutdown_connections 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.491+0000 7f958a5fa700 1 -- 172.21.15.62:0/3703096469 wait complete. 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f958a5fa700 1 Processor -- start 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f958a5fa700 1 -- start start 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f958a5fa700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95840ffd80 0x7f9584103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f958a5fa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9584100f70 0x7f9584102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f958a5fa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9584101910 0x7f9584102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f958a5fa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9584103030 con 0x7f9584100f70 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f958a5fa700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95841031a0 con 0x7f95840ffd80 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f958a5fa700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95841a0e60 con 0x7f9584101910 2024-05-08T22:38:34.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f95837fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9584100f70 0x7f9584102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f95837fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9584100f70 0x7f9584102350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58904/0 (socket says 172.21.15.62:58904) 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f95837fe700 1 -- 172.21.15.62:0/1145021740 learned_addr learned my addr 172.21.15.62:0/1145021740 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f9588b97700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9584101910 0x7f9584102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.492+0000 7f9583fff700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95840ffd80 0x7f9584103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.493+0000 7f95837fe700 1 -- 172.21.15.62:0/1145021740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9584101910 msgr2=0x7f9584102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.493+0000 7f95837fe700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9584101910 0x7f9584102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.493+0000 7f95837fe700 1 -- 172.21.15.62:0/1145021740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95840ffd80 msgr2=0x7f9584103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.493+0000 7f95837fe700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95840ffd80 0x7f9584103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.493+0000 7f95837fe700 1 -- 172.21.15.62:0/1145021740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95740095f0 con 0x7f9584100f70 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.493+0000 7f95837fe700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9584100f70 0x7f9584102350 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f957800e9a0 tx=0x7f957800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.493+0000 7f95817fa700 1 -- 172.21.15.62:0/1145021740 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578009790 con 0x7f9584100f70 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.493+0000 7f95817fa700 1 -- 172.21.15.62:0/1145021740 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f957800ce70 con 0x7f9584100f70 2024-05-08T22:38:34.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.493+0000 7f958a5fa700 1 -- 172.21.15.62:0/1145021740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95841a10f0 con 0x7f9584100f70 2024-05-08T22:38:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.493+0000 7f95817fa700 1 -- 172.21.15.62:0/1145021740 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578018420 con 0x7f9584100f70 2024-05-08T22:38:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.493+0000 7f958a5fa700 1 -- 172.21.15.62:0/1145021740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95841a1650 con 0x7f9584100f70 2024-05-08T22:38:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.494+0000 7f958a5fa700 1 -- 172.21.15.62:0/1145021740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9584067d10 con 0x7f9584100f70 2024-05-08T22:38:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.495+0000 7f95817fa700 1 -- 172.21.15.62:0/1145021740 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f95780106f0 con 0x7f9584100f70 2024-05-08T22:38:34.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.496+0000 7f95817fa700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f956c0a11a0 0x7f956c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.496+0000 7f9583fff700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f956c0a11a0 0x7f956c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.496+0000 7f95817fa700 1 -- 172.21.15.62:0/1145021740 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9578014070 con 0x7f9584100f70 2024-05-08T22:38:34.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.496+0000 7f9583fff700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f956c0a11a0 0x7f956c0a3660 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9574009fd0 tx=0x7f9574019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:34.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.499+0000 7f95817fa700 1 -- 172.21.15.62:0/1145021740 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f95780965d0 con 0x7f9584100f70 2024-05-08T22:38:34.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.755+0000 7f958a5fa700 1 -- 172.21.15.62:0/1145021740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f958410b590 con 0x7f9584100f70 2024-05-08T22:38:34.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.756+0000 7f95817fa700 1 -- 172.21.15.62:0/1145021740 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10899 (secure 0 0 0) 0x7f9578026090 con 0x7f9584100f70 2024-05-08T22:38:34.755 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:38:34.755 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3584665258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3584665258},{"type":"v1","addr":"172.21.15.62:6835","nonce":3584665258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2939636235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2939636235},{"type":"v1","addr":"172.21.15.62:6837","nonce":2939636235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T22:35:09.803459+0000","modified":"2024-05-08T22:38:24.738277+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24529,"mds_2":24505,"mds_3":24503,"mds_4":14661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/1756743060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":1756743060},{"type":"v1","addr":"172.21.15.62:6839","nonce":1756743060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14661":{"gid":14661,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.62:6841/780017086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":780017086},{"type":"v1","addr":"172.21.15.62:6841","nonce":780017086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/4065089721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4065089721},{"type":"v1","addr":"172.21.15.66:6833","nonce":4065089721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6833/1379725080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1379725080},{"type":"v1","addr":"172.21.15.79:6833","nonce":1379725080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6835/2449812673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2449812673},{"type":"v1","addr":"172.21.15.66:6835","nonce":2449812673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/2938225588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2938225588},{"type":"v1","addr":"172.21.15.79:6835","nonce":2938225588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/1178845415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1178845415},{"type":"v1","addr":"172.21.15.66:6837","nonce":1178845415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.79:6837/753714978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":753714978},{"type":"v1","addr":"172.21.15.79:6837","nonce":753714978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.79:6839/3534785027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":3534785027},{"type":"v1","addr":"172.21.15.79:6839","nonce":3534785027}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/3474672912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":3474672912},{"type":"v1","addr":"172.21.15.66:6839","nonce":3474672912}]},"join_fscid":-1,"export_targets":[4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T22:38:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 -- 172.21.15.62:0/1145021740 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f956c0a11a0 msgr2=0x7f956c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f956c0a11a0 0x7f956c0a3660 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9574009fd0 tx=0x7f9574019040 comp rx=0 tx=0).stop 2024-05-08T22:38:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 -- 172.21.15.62:0/1145021740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9584100f70 msgr2=0x7f9584102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9584100f70 0x7f9584102350 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f957800e9a0 tx=0x7f957800ecb0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 -- 172.21.15.62:0/1145021740 shutdown_connections 2024-05-08T22:38:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9584101910 0x7f9584102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f956c0a11a0 0x7f956c0a3660 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95840ffd80 0x7f9584103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 --2- 172.21.15.62:0/1145021740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9584100f70 0x7f9584102350 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 -- 172.21.15.62:0/1145021740 >> 172.21.15.62:0/1145021740 conn(0x7f95840fb310 msgr2=0x7f9584068190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 -- 172.21.15.62:0/1145021740 shutdown_connections 2024-05-08T22:38:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.758+0000 7f958a5fa700 1 -- 172.21.15.62:0/1145021740 wait complete. 2024-05-08T22:38:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 31 2024-05-08T22:38:34.771 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T22:38:34.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.900+0000 7f55cae2c700 1 -- 172.21.15.62:0/1739014786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c40ffd80 msgr2=0x7f55c41001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.900+0000 7f55cae2c700 1 --2- 172.21.15.62:0/1739014786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c40ffd80 0x7f55c41001a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f55b80099c0 tx=0x7f55b8009cd0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.900+0000 7f55cae2c700 1 -- 172.21.15.62:0/1739014786 shutdown_connections 2024-05-08T22:38:34.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.900+0000 7f55cae2c700 1 --2- 172.21.15.62:0/1739014786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55c4101910 0x7f55c41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.900+0000 7f55cae2c700 1 --2- 172.21.15.62:0/1739014786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c40ffd80 0x7f55c41001a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.900+0000 7f55cae2c700 1 --2- 172.21.15.62:0/1739014786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4100f70 0x7f55c41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.900+0000 7f55cae2c700 1 -- 172.21.15.62:0/1739014786 >> 172.21.15.62:0/1739014786 conn(0x7f55c40fb2f0 msgr2=0x7f55c40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:34.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.900+0000 7f55cae2c700 1 -- 172.21.15.62:0/1739014786 shutdown_connections 2024-05-08T22:38:34.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.900+0000 7f55cae2c700 1 -- 172.21.15.62:0/1739014786 wait complete. 2024-05-08T22:38:34.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55cae2c700 1 Processor -- start 2024-05-08T22:38:34.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55cae2c700 1 -- start start 2024-05-08T22:38:34.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55cae2c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c40ffd80 0x7f55c4103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55cae2c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4100f70 0x7f55c4102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55cae2c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55c4101910 0x7f55c4102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55cae2c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c4103030 con 0x7f55c40ffd80 2024-05-08T22:38:34.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55cae2c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c41031a0 con 0x7f55c4101910 2024-05-08T22:38:34.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55cae2c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c41a0f20 con 0x7f55c4100f70 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c8bc8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c40ffd80 0x7f55c4103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c8bc8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c40ffd80 0x7f55c4103cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58916/0 (socket says 172.21.15.62:58916) 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c8bc8700 1 -- 172.21.15.62:0/923762789 learned_addr learned my addr 172.21.15.62:0/923762789 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c3fff700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4100f70 0x7f55c4102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c93c9700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55c4101910 0x7f55c4102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c8bc8700 1 -- 172.21.15.62:0/923762789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4100f70 msgr2=0x7f55c4102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c8bc8700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4100f70 0x7f55c4102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c8bc8700 1 -- 172.21.15.62:0/923762789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55c4101910 msgr2=0x7f55c4102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c8bc8700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55c4101910 0x7f55c4102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c8bc8700 1 -- 172.21.15.62:0/923762789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b80095f0 con 0x7f55c40ffd80 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c93c9700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55c4101910 0x7f55c4102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.901+0000 7f55c8bc8700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c40ffd80 0x7f55c4103cc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f55b8006010 tx=0x7f55b8005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:34.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.902+0000 7f55c1ffb700 1 -- 172.21.15.62:0/923762789 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b801d070 con 0x7f55c40ffd80 2024-05-08T22:38:34.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.902+0000 7f55c1ffb700 1 -- 172.21.15.62:0/923762789 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55b8004150 con 0x7f55c40ffd80 2024-05-08T22:38:34.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.902+0000 7f55cae2c700 1 -- 172.21.15.62:0/923762789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c41a1150 con 0x7f55c40ffd80 2024-05-08T22:38:34.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.902+0000 7f55c1ffb700 1 -- 172.21.15.62:0/923762789 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800f460 con 0x7f55c40ffd80 2024-05-08T22:38:34.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.902+0000 7f55cae2c700 1 -- 172.21.15.62:0/923762789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c41a1510 con 0x7f55c40ffd80 2024-05-08T22:38:34.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.903+0000 7f55cae2c700 1 -- 172.21.15.62:0/923762789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c4067480 con 0x7f55c40ffd80 2024-05-08T22:38:34.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.904+0000 7f55c1ffb700 1 -- 172.21.15.62:0/923762789 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f55b8022700 con 0x7f55c40ffd80 2024-05-08T22:38:34.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.904+0000 7f55c1ffb700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f55b40a1120 0x7f55b40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:34.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.905+0000 7f55c1ffb700 1 -- 172.21.15.62:0/923762789 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f55b80cdfa0 con 0x7f55c40ffd80 2024-05-08T22:38:34.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.905+0000 7f55c3fff700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f55b40a1120 0x7f55b40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:34.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.905+0000 7f55c3fff700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f55b40a1120 0x7f55b40a35e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f55b000bbd0 tx=0x7f55b000b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:34.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:34.908+0000 7f55c1ffb700 1 -- 172.21.15.62:0/923762789 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f55b8097870 con 0x7f55c40ffd80 2024-05-08T22:38:35.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:34 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/63744544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:38:35.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:34 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/1145021740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:38:35.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.117+0000 7f55cae2c700 1 -- 172.21.15.62:0/923762789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55c410b5e0 con 0x7f55c40ffd80 2024-05-08T22:38:35.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.119+0000 7f55c1ffb700 1 -- 172.21.15.62:0/923762789 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22481 (secure 0 0 0) 0x7f55b809ae90 con 0x7f55c40ffd80 2024-05-08T22:38:35.118 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-05-08T22:38:35.118 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"49d88ea8-0d8a-11ef-bc97-c7b262605968","created":"2024-05-08T22:29:16.808489+0000","modified":"2024-05-08T22:36:05.960694+0000","last_up_change":"2024-05-08T22:34:34.349875+0000","last_in_change":"2024-05-08T22:34:16.995052+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T22:31:54.466249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T22:35:08.296334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T22:35:09.167901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7242466d-5961-4fd3-8d3e-2996fcccd92e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6803","nonce":2221009645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6805","nonce":2221009645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6809","nonce":2221009645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2221009645},{"type":"v1","addr":"172.21.15.62:6807","nonce":2221009645}]},"public_addr":"172.21.15.62:6803/2221009645","cluster_addr":"172.21.15.62:6805/2221009645","heartbeat_back_addr":"172.21.15.62:6809/2221009645","heartbeat_front_addr":"172.21.15.62:6807/2221009645","state":["exists","up"]},{"osd":1,"uuid":"3e358e76-c55b-4e94-a8e6-727c65c10718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6801","nonce":1982485054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6803","nonce":1982485054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6807","nonce":1982485054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1982485054},{"type":"v1","addr":"172.21.15.66:6805","nonce":1982485054}]},"public_addr":"172.21.15.66:6801/1982485054","cluster_addr":"172.21.15.66:6803/1982485054","heartbeat_back_addr":"172.21.15.66:6807/1982485054","heartbeat_front_addr":"172.21.15.66:6805/1982485054","state":["exists","up"]},{"osd":2,"uuid":"4a8002d5-c2cc-4275-aa59-bb335b2b8740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6801","nonce":2543768835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6803","nonce":2543768835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6807","nonce":2543768835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2543768835},{"type":"v1","addr":"172.21.15.79:6805","nonce":2543768835}]},"public_addr":"172.21.15.79:6801/2543768835","cluster_addr":"172.21.15.79:6803/2543768835","heartbeat_back_addr":"172.21.15.79:6807/2543768835","heartbeat_front_addr":"172.21.15.79:6805/2543768835","state":["exists","up"]},{"osd":3,"uuid":"2cb8e9ce-ae6a-4c93-a888-2e0c831691d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6811","nonce":2546077230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6813","nonce":2546077230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6817","nonce":2546077230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2546077230},{"type":"v1","addr":"172.21.15.62:6815","nonce":2546077230}]},"public_addr":"172.21.15.62:6811/2546077230","cluster_addr":"172.21.15.62:6813/2546077230","heartbeat_back_addr":"172.21.15.62:6817/2546077230","heartbeat_front_addr":"172.21.15.62:6815/2546077230","state":["exists","up"]},{"osd":4,"uuid":"7d3e6d05-7c0c-4098-8e91-e91fce3173cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6809","nonce":2311123866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6811","nonce":2311123866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6815","nonce":2311123866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2311123866},{"type":"v1","addr":"172.21.15.66:6813","nonce":2311123866}]},"public_addr":"172.21.15.66:6809/2311123866","cluster_addr":"172.21.15.66:6811/2311123866","heartbeat_back_addr":"172.21.15.66:6815/2311123866","heartbeat_front_addr":"172.21.15.66:6813/2311123866","state":["exists","up"]},{"osd":5,"uuid":"e7f834ac-4a24-4111-a581-4276baaa08a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6809","nonce":1027194211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6811","nonce":1027194211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6815","nonce":1027194211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1027194211},{"type":"v1","addr":"172.21.15.79:6813","nonce":1027194211}]},"public_addr":"172.21.15.79:6809/1027194211","cluster_addr":"172.21.15.79:6811/1027194211","heartbeat_back_addr":"172.21.15.79:6815/1027194211","heartbeat_front_addr":"172.21.15.79:6813/1027194211","state":["exists","up"]},{"osd":6,"uuid":"da290a10-42f6-4e73-a6e6-e716af055721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6819","nonce":1217656145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6821","nonce":1217656145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6825","nonce":1217656145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1217656145},{"type":"v1","addr":"172.21.15.62:6823","nonce":1217656145}]},"public_addr":"172.21.15.62:6819/1217656145","cluster_addr":"172.21.15.62:6821/1217656145","heartbeat_back_addr":"172.21.15.62:6825/1217656145","heartbeat_front_addr":"172.21.15.62:6823/1217656145","state":["exists","up"]},{"osd":7,"uuid":"ee23b28b-603a-4f50-b3f1-2404bef4fa74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6817","nonce":653963757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6819","nonce":653963757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6823","nonce":653963757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":653963757},{"type":"v1","addr":"172.21.15.66:6821","nonce":653963757}]},"public_addr":"172.21.15.66:6817/653963757","cluster_addr":"172.21.15.66:6819/653963757","heartbeat_back_addr":"172.21.15.66:6823/653963757","heartbeat_front_addr":"172.21.15.66:6821/653963757","state":["exists","up"]},{"osd":8,"uuid":"5f193ce6-5093-45c8-aee0-2c6e99873111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6817","nonce":2871988040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6819","nonce":2871988040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6823","nonce":2871988040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2871988040},{"type":"v1","addr":"172.21.15.79:6821","nonce":2871988040}]},"public_addr":"172.21.15.79:6817/2871988040","cluster_addr":"172.21.15.79:6819/2871988040","heartbeat_back_addr":"172.21.15.79:6823/2871988040","heartbeat_front_addr":"172.21.15.79:6821/2871988040","state":["exists","up"]},{"osd":9,"uuid":"1811353e-323e-4f18-8c1b-5121e896aa68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6827","nonce":1806941351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6829","nonce":1806941351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6833","nonce":1806941351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1806941351},{"type":"v1","addr":"172.21.15.62:6831","nonce":1806941351}]},"public_addr":"172.21.15.62:6827/1806941351","cluster_addr":"172.21.15.62:6829/1806941351","heartbeat_back_addr":"172.21.15.62:6833/1806941351","heartbeat_front_addr":"172.21.15.62:6831/1806941351","state":["exists","up"]},{"osd":10,"uuid":"e1769ba4-8dd8-4fb2-a596-e37d465a58d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6825","nonce":1104097413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6827","nonce":1104097413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6831","nonce":1104097413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1104097413},{"type":"v1","addr":"172.21.15.66:6829","nonce":1104097413}]},"public_addr":"172.21.15.66:6825/1104097413","cluster_addr":"172.21.15.66:6827/1104097413","heartbeat_back_addr":"172.21.15.66:6831/1104097413","heartbeat_front_addr":"172.21.15.66:6829/1104097413","state":["exists","up"]},{"osd":11,"uuid":"5d7d0579-5d22-46e8-a43b-714675f3dd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6825","nonce":1561471078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6827","nonce":1561471078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6831","nonce":1561471078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1561471078},{"type":"v1","addr":"172.21.15.79:6829","nonce":1561471078}]},"public_addr":"172.21.15.79:6825/1561471078","cluster_addr":"172.21.15.79:6827/1561471078","heartbeat_back_addr":"172.21.15.79:6831/1561471078","heartbeat_front_addr":"172.21.15.79:6829/1561471078","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:16.281079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:34.107797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:31:51.934155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:10.036130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:27.579911+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:32:45.211154+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:03.037675+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:20.233331+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:38.261590+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:33:56.223591+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:14.310284+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T22:34:31.896933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:6800/766518770":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2477775765":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/2433771662":"2024-05-09T22:30:03.951072+0000","172.21.15.62:0/1318760241":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/1247031049":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6800/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:6801/3409257313":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/3020122046":"2024-05-09T22:29:43.058069+0000","172.21.15.62:0/2719135336":"2024-05-09T22:30:03.951072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T22:38:35.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 -- 172.21.15.62:0/923762789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f55b40a1120 msgr2=0x7f55b40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:35.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f55b40a1120 0x7f55b40a35e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f55b000bbd0 tx=0x7f55b000b5b0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 -- 172.21.15.62:0/923762789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c40ffd80 msgr2=0x7f55c4103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:35.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c40ffd80 0x7f55c4103cc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f55b8006010 tx=0x7f55b8005af0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 -- 172.21.15.62:0/923762789 shutdown_connections 2024-05-08T22:38:35.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f55b40a1120 0x7f55b40a35e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55c4101910 0x7f55c4102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c40ffd80 0x7f55c4103cc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 --2- 172.21.15.62:0/923762789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4100f70 0x7f55c4102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 -- 172.21.15.62:0/923762789 >> 172.21.15.62:0/923762789 conn(0x7f55c40fb2f0 msgr2=0x7f55c40690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 -- 172.21.15.62:0/923762789 shutdown_connections 2024-05-08T22:38:35.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.121+0000 7f55cae2c700 1 -- 172.21.15.62:0/923762789 wait complete. 2024-05-08T22:38:35.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:34 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/63744544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:38:35.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:34 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/1145021740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:38:35.135 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T22:38:35.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/63744544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:38:35.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:34 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/1145021740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T22:38:35.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.263+0000 7f089d0a5700 1 -- 172.21.15.62:0/2651679194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0898100fb0 msgr2=0x7f0898101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.263+0000 7f089d0a5700 1 --2- 172.21.15.62:0/2651679194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0898100fb0 0x7f0898101410 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f088c0099c0 tx=0x7f088c009cd0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.263+0000 7f089d0a5700 1 -- 172.21.15.62:0/2651679194 shutdown_connections 2024-05-08T22:38:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.263+0000 7f089d0a5700 1 --2- 172.21.15.62:0/2651679194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0898101950 0x7f0898108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.263+0000 7f089d0a5700 1 --2- 172.21.15.62:0/2651679194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08980ffdc0 0x7f08981001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.263+0000 7f089d0a5700 1 --2- 172.21.15.62:0/2651679194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0898100fb0 0x7f0898101410 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.263+0000 7f089d0a5700 1 -- 172.21.15.62:0/2651679194 >> 172.21.15.62:0/2651679194 conn(0x7f08980fb330 msgr2=0x7f08980fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.263+0000 7f089d0a5700 1 -- 172.21.15.62:0/2651679194 shutdown_connections 2024-05-08T22:38:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.264+0000 7f089d0a5700 1 -- 172.21.15.62:0/2651679194 wait complete. 2024-05-08T22:38:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.264+0000 7f089d0a5700 1 Processor -- start 2024-05-08T22:38:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.264+0000 7f089d0a5700 1 -- start start 2024-05-08T22:38:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f089d0a5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08980ffdc0 0x7f0898069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f089d0a5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0898100fb0 0x7f089819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f089d0a5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0898101950 0x7f089819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f089d0a5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089819e8d0 con 0x7f08980ffdc0 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f089d0a5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089819ea40 con 0x7f0898101950 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f089d0a5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089819ec00 con 0x7f0898100fb0 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f0896d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08980ffdc0 0x7f0898069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f0896d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08980ffdc0 0x7f0898069b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58928/0 (socket says 172.21.15.62:58928) 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f0896d9d700 1 -- 172.21.15.62:0/872213781 learned_addr learned my addr 172.21.15.62:0/872213781 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f089659c700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0898100fb0 0x7f089819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f089759e700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0898101950 0x7f089819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f0896d9d700 1 -- 172.21.15.62:0/872213781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0898100fb0 msgr2=0x7f089819bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f0896d9d700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0898100fb0 0x7f089819bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f0896d9d700 1 -- 172.21.15.62:0/872213781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0898101950 msgr2=0x7f089819e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f0896d9d700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0898101950 0x7f089819e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f0896d9d700 1 -- 172.21.15.62:0/872213781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0095f0 con 0x7f08980ffdc0 2024-05-08T22:38:35.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f089759e700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0898101950 0x7f089819e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T22:38:35.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f0896d9d700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08980ffdc0 0x7f0898069b70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f088800d980 tx=0x7f088800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:35.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f087ffff700 1 -- 172.21.15.62:0/872213781 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888004020 con 0x7f08980ffdc0 2024-05-08T22:38:35.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f087ffff700 1 -- 172.21.15.62:0/872213781 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0888009730 con 0x7f08980ffdc0 2024-05-08T22:38:35.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f089d0a5700 1 -- 172.21.15.62:0/872213781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089819eee0 con 0x7f08980ffdc0 2024-05-08T22:38:35.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.265+0000 7f087ffff700 1 -- 172.21.15.62:0/872213781 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888003a80 con 0x7f08980ffdc0 2024-05-08T22:38:35.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.266+0000 7f089d0a5700 1 -- 172.21.15.62:0/872213781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089819f310 con 0x7f08980ffdc0 2024-05-08T22:38:35.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.266+0000 7f089d0a5700 1 -- 172.21.15.62:0/872213781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0898107230 con 0x7f08980ffdc0 2024-05-08T22:38:35.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.267+0000 7f087ffff700 1 -- 172.21.15.62:0/872213781 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f08880098a0 con 0x7f08980ffdc0 2024-05-08T22:38:35.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.268+0000 7f087ffff700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f08800a1120 0x7f08800a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:35.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.268+0000 7f089659c700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f08800a1120 0x7f08800a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:35.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.269+0000 7f087ffff700 1 -- 172.21.15.62:0/872213781 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08880cc920 con 0x7f08980ffdc0 2024-05-08T22:38:35.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.269+0000 7f089659c700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f08800a1120 0x7f08800a35e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f088c009990 tx=0x7f088c005ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.272+0000 7f087ffff700 1 -- 172.21.15.62:0/872213781 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f08880960c0 con 0x7f08980ffdc0 2024-05-08T22:38:35.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.532+0000 7f089d0a5700 1 -- 172.21.15.62:0/872213781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f089810b5d0 con 0x7f08980ffdc0 2024-05-08T22:38:35.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.546+0000 7f087ffff700 1 -- 172.21.15.62:0/872213781 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) v1 ==== 161+0+0 (secure 0 0 0) 0x7f08880996e0 con 0x7f08980ffdc0 2024-05-08T22:38:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 -- 172.21.15.62:0/872213781 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f08800a1120 msgr2=0x7f08800a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f08800a1120 0x7f08800a35e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f088c009990 tx=0x7f088c005ad0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 -- 172.21.15.62:0/872213781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08980ffdc0 msgr2=0x7f0898069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08980ffdc0 0x7f0898069b70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f088800d980 tx=0x7f088800dc90 comp rx=0 tx=0).stop 2024-05-08T22:38:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 -- 172.21.15.62:0/872213781 shutdown_connections 2024-05-08T22:38:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f08800a1120 0x7f08800a35e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0898101950 0x7f089819e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08980ffdc0 0x7f0898069b70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 --2- 172.21.15.62:0/872213781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0898100fb0 0x7f089819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 -- 172.21.15.62:0/872213781 >> 172.21.15.62:0/872213781 conn(0x7f08980fb330 msgr2=0x7f0898104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 -- 172.21.15.62:0/872213781 shutdown_connections 2024-05-08T22:38:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.548+0000 7f089d0a5700 1 -- 172.21.15.62:0/872213781 wait complete. 2024-05-08T22:38:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T22:38:35.562 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T22:38:35.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.696+0000 7f6b0ca41700 1 -- 172.21.15.62:0/3025574788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b081000f0 msgr2=0x7f6b08100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.696+0000 7f6b0ca41700 1 --2- 172.21.15.62:0/3025574788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b081000f0 0x7f6b08100570 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009990 tx=0x7f6af0009ca0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.696+0000 7f6b0ca41700 1 -- 172.21.15.62:0/3025574788 shutdown_connections 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.696+0000 7f6b0ca41700 1 --2- 172.21.15.62:0/3025574788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b08100ab0 0x7f6b08100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.696+0000 7f6b0ca41700 1 --2- 172.21.15.62:0/3025574788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b080ff790 0x7f6b080ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.696+0000 7f6b0ca41700 1 --2- 172.21.15.62:0/3025574788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b081000f0 0x7f6b08100570 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.696+0000 7f6b0ca41700 1 -- 172.21.15.62:0/3025574788 >> 172.21.15.62:0/3025574788 conn(0x7f6b080fb2d0 msgr2=0x7f6b080fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.696+0000 7f6b0ca41700 1 -- 172.21.15.62:0/3025574788 shutdown_connections 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.696+0000 7f6b0ca41700 1 -- 172.21.15.62:0/3025574788 wait complete. 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.697+0000 7f6b0ca41700 1 Processor -- start 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.697+0000 7f6b0ca41700 1 -- start start 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.697+0000 7f6b0ca41700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b080ff790 0x7f6b08198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.697+0000 7f6b0ca41700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b081000f0 0x7f6b08199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.697+0000 7f6b0ca41700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b08100ab0 0x7f6b081a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.697+0000 7f6b0ca41700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b081a0b10 con 0x7f6b080ff790 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.697+0000 7f6b0ca41700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b081a0c50 con 0x7f6b081000f0 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.697+0000 7f6b0ca41700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b081a0de0 con 0x7f6b08100ab0 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.697+0000 7f6b0659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b080ff790 0x7f6b08198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b05d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b081000f0 0x7f6b08199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b0659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b080ff790 0x7f6b08198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58984/0 (socket says 172.21.15.62:58984) 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b0659c700 1 -- 172.21.15.62:0/2577518990 learned_addr learned my addr 172.21.15.62:0/2577518990 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b06d9d700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b08100ab0 0x7f6b081a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b05d9b700 1 -- 172.21.15.62:0/2577518990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b08100ab0 msgr2=0x7f6b081a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b05d9b700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b08100ab0 0x7f6b081a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b05d9b700 1 -- 172.21.15.62:0/2577518990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b080ff790 msgr2=0x7f6b08198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b05d9b700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b080ff790 0x7f6b08198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b05d9b700 1 -- 172.21.15.62:0/2577518990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00095f0 con 0x7f6b081000f0 2024-05-08T22:38:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b0659c700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b080ff790 0x7f6b08198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T22:38:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b05d9b700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b081000f0 0x7f6b08199520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6af0006010 tx=0x7f6af0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6aff7fe700 1 -- 172.21.15.62:0/2577518990 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af001d070 con 0x7f6b081000f0 2024-05-08T22:38:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b0ca41700 1 -- 172.21.15.62:0/2577518990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b081a1060 con 0x7f6b081000f0 2024-05-08T22:38:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6aff7fe700 1 -- 172.21.15.62:0/2577518990 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6af00054f0 con 0x7f6b081000f0 2024-05-08T22:38:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6aff7fe700 1 -- 172.21.15.62:0/2577518990 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0004cd0 con 0x7f6b081000f0 2024-05-08T22:38:35.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.698+0000 7f6b0ca41700 1 -- 172.21.15.62:0/2577518990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b081a1420 con 0x7f6b081000f0 2024-05-08T22:38:35.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.699+0000 7f6b0ca41700 1 -- 172.21.15.62:0/2577518990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0810a570 con 0x7f6b081000f0 2024-05-08T22:38:35.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.700+0000 7f6aff7fe700 1 -- 172.21.15.62:0/2577518990 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6af0005030 con 0x7f6b081000f0 2024-05-08T22:38:35.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.701+0000 7f6aff7fe700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6af40a1120 0x7f6af40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:35.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.701+0000 7f6b0659c700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6af40a1120 0x7f6af40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:35.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.701+0000 7f6aff7fe700 1 -- 172.21.15.62:0/2577518990 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8641+0+0 (secure 0 0 0) 0x7f6af00cdf30 con 0x7f6b081000f0 2024-05-08T22:38:35.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.701+0000 7f6b0659c700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6af40a1120 0x7f6af40a35e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009c10 tx=0x7f6af800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:35.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.705+0000 7f6aff7fe700 1 -- 172.21.15.62:0/2577518990 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6af00975b0 con 0x7f6b081000f0 2024-05-08T22:38:35.888 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-c[84834]: ignoring --setuser ceph since I am not root 2024-05-08T22:38:35.889 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-c[84834]: ignoring --setgroup ceph since I am not root 2024-05-08T22:38:35.889 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-c[84834]: starting mds.c at 2024-05-08T22:38:35.889 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-i[85180]: ignoring --setuser ceph since I am not root 2024-05-08T22:38:35.889 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-i[85180]: ignoring --setgroup ceph since I am not root 2024-05-08T22:38:35.889 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-i[85180]: starting mds.i at 2024-05-08T22:38:35.889 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-l[84138]: ignoring --setuser ceph since I am not root 2024-05-08T22:38:35.890 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-l[84138]: ignoring --setgroup ceph since I am not root 2024-05-08T22:38:35.890 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 08 22:38:35 smithi079 ceph-mds[84161]: -- 172.21.15.79:0/154163283 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e7ee57a1a0 con 0x55e7ee550800 2024-05-08T22:38:35.890 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-l[84138]: starting mds.l at 2024-05-08T22:38:35.890 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-f[84487]: ignoring --setuser ceph since I am not root 2024-05-08T22:38:35.890 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-f[84487]: ignoring --setgroup ceph since I am not root 2024-05-08T22:38:35.890 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:38:35 smithi079 ceph-mds[84510]: -- 172.21.15.79:0/250457752 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5580fd7301a0 con 0x5580fd706c00 2024-05-08T22:38:35.890 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:38:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-f[84487]: starting mds.f at 2024-05-08T22:38:35.890 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:35 smithi079 ceph-mon[46738]: pgmap v326: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-08T22:38:35.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:35 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/923762789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:38:35.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:35 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/872213781' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:38:35.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:35 smithi079 ceph-mon[46738]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T22:38:35.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:35 smithi079 ceph-mon[46738]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:38:35.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:35 smithi079 ceph-mon[46738]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T22:38:35.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:35 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/872213781' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T22:38:35.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:35 smithi079 ceph-mon[46738]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T22:38:35.921 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:38:35 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-g[95555]: ignoring --setuser ceph since I am not root 2024-05-08T22:38:35.921 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:38:35 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-g[95555]: ignoring --setgroup ceph since I am not root 2024-05-08T22:38:35.921 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:38:35 smithi062 ceph-mds[95578]: -- 172.21.15.62:0/3226933934 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5623ae2321a0 con 0x5623ae208800 2024-05-08T22:38:35.921 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:38:35 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-g[95555]: starting mds.g at 2024-05-08T22:38:35.922 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-a[95945]: ignoring --setuser ceph since I am not root 2024-05-08T22:38:35.922 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-a[95945]: ignoring --setgroup ceph since I am not root 2024-05-08T22:38:35.922 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-mds[95968]: -- 172.21.15.62:0/596581336 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5627a38b01a0 con 0x5627a3886800 2024-05-08T22:38:35.922 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-a[95945]: starting mds.a at 2024-05-08T22:38:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-a[42548]: 2024-05-08T22:38:35.533+0000 7f1e82949700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:38:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-mon[42571]: pgmap v326: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-08T22:38:35.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/923762789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:38:35.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/872213781' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:38:35.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-mon[42571]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T22:38:35.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-mon[42571]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:38:35.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-mon[42571]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T22:38:35.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/872213781' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T22:38:35.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:35 smithi062 ceph-mon[42571]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T22:38:35.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:35.962+0000 7f6b0ca41700 1 -- 172.21.15.62:0/2577518990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6b0810e9a0 con 0x7f6b081000f0 2024-05-08T22:38:36.073 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-b[83604]: ignoring --setuser ceph since I am not root 2024-05-08T22:38:36.073 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-b[83604]: ignoring --setgroup ceph since I am not root 2024-05-08T22:38:36.073 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-mds[83627]: -- 172.21.15.66:0/2071585260 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560de5e1c1a0 con 0x560de5df2c00 2024-05-08T22:38:36.073 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-b[83604]: starting mds.b at 2024-05-08T22:38:36.074 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-h[83262]: ignoring --setuser ceph since I am not root 2024-05-08T22:38:36.074 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-h[83262]: ignoring --setgroup ceph since I am not root 2024-05-08T22:38:36.074 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-h[83262]: starting mds.h at 2024-05-08T22:38:36.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-mon[46830]: pgmap v326: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-08T22:38:36.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/923762789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T22:38:36.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/872213781' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T22:38:36.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-mon[46830]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T22:38:36.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-mon[46830]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T22:38:36.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-mon[46830]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T22:38:36.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/872213781' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T22:38:36.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:35 smithi066 ceph-mon[46830]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T22:38:36.075 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-e[82921]: ignoring --setuser ceph since I am not root 2024-05-08T22:38:36.075 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-e[82921]: ignoring --setgroup ceph since I am not root 2024-05-08T22:38:36.075 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 08 22:38:35 smithi066 ceph-mds[82944]: -- 172.21.15.66:0/2513623521 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eba7ab01a0 con 0x55eba7a86800 2024-05-08T22:38:36.075 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-e[82921]: starting mds.e at 2024-05-08T22:38:36.076 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-k[82581]: ignoring --setuser ceph since I am not root 2024-05-08T22:38:36.076 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-k[82581]: ignoring --setgroup ceph since I am not root 2024-05-08T22:38:36.076 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:38:35 smithi066 ceph-mds[82604]: -- 172.21.15.66:0/3307341426 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a0a07b61a0 con 0x55a0a078d000 2024-05-08T22:38:36.076 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:38:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-k[82581]: starting mds.k at 2024-05-08T22:38:36.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.548+0000 7f6aff7fe700 1 -- 172.21.15.62:0/2577518990 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6af009abd0 con 0x7f6b081000f0 2024-05-08T22:38:36.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.552+0000 7f6b0ca41700 1 -- 172.21.15.62:0/2577518990 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6af40a1120 msgr2=0x7f6af40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:36.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.552+0000 7f6b0ca41700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6af40a1120 0x7f6af40a35e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009c10 tx=0x7f6af800b040 comp rx=0 tx=0).stop 2024-05-08T22:38:36.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.552+0000 7f6b0ca41700 1 -- 172.21.15.62:0/2577518990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b081000f0 msgr2=0x7f6b08199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:36.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.552+0000 7f6b0ca41700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b081000f0 0x7f6b08199520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6af0006010 tx=0x7f6af0004730 comp rx=0 tx=0).stop 2024-05-08T22:38:36.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.553+0000 7f6b0ca41700 1 -- 172.21.15.62:0/2577518990 shutdown_connections 2024-05-08T22:38:36.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.553+0000 7f6b0ca41700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b08100ab0 0x7f6b081a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:36.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.553+0000 7f6b0ca41700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b080ff790 0x7f6b08198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:36.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.553+0000 7f6b0ca41700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6af40a1120 0x7f6af40a35e0 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009c10 tx=0x7f6af800b040 comp rx=0 tx=0).stop 2024-05-08T22:38:36.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.553+0000 7f6b0ca41700 1 --2- 172.21.15.62:0/2577518990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b081000f0 0x7f6b08199520 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:36.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.553+0000 7f6b0ca41700 1 -- 172.21.15.62:0/2577518990 >> 172.21.15.62:0/2577518990 conn(0x7f6b080fb2d0 msgr2=0x7f6b080fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:36.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.554+0000 7f6b0ca41700 1 -- 172.21.15.62:0/2577518990 shutdown_connections 2024-05-08T22:38:36.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.554+0000 7f6b0ca41700 1 -- 172.21.15.62:0/2577518990 wait complete. 2024-05-08T22:38:36.569 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T22:38:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.708+0000 7f7085567700 1 -- 172.21.15.62:0/2352081258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7080101920 msgr2=0x7f70801081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.708+0000 7f7085567700 1 --2- 172.21.15.62:0/2352081258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7080101920 0x7f70801081d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f707800b0d0 tx=0x7f707800b3e0 comp rx=0 tx=0).stop 2024-05-08T22:38:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.708+0000 7f7085567700 1 -- 172.21.15.62:0/2352081258 shutdown_connections 2024-05-08T22:38:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.708+0000 7f7085567700 1 --2- 172.21.15.62:0/2352081258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7080101920 0x7f70801081d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.708+0000 7f7085567700 1 --2- 172.21.15.62:0/2352081258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70800ffd90 0x7f70801001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.708+0000 7f7085567700 1 --2- 172.21.15.62:0/2352081258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7080100f80 0x7f70801013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.708+0000 7f7085567700 1 -- 172.21.15.62:0/2352081258 >> 172.21.15.62:0/2352081258 conn(0x7f70800fb340 msgr2=0x7f70800fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.708+0000 7f7085567700 1 -- 172.21.15.62:0/2352081258 shutdown_connections 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.708+0000 7f7085567700 1 -- 172.21.15.62:0/2352081258 wait complete. 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f7085567700 1 Processor -- start 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f7085567700 1 -- start start 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f7085567700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800ffd90 0x7f708006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f7085567700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7080100f80 0x7f708019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f7085567700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7080101920 0x7f70801a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f7085567700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70801a0c00 con 0x7f70800ffd90 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f7085567700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70801a0d70 con 0x7f7080100f80 2024-05-08T22:38:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f7085567700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70801a0f30 con 0x7f7080101920 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f707effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800ffd90 0x7f708006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f707effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800ffd90 0x7f708006a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38704/0 (socket says 172.21.15.62:38704) 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f707effd700 1 -- 172.21.15.62:0/4490574 learned_addr learned my addr 172.21.15.62:0/4490574 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f707f7fe700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7080101920 0x7f70801a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.709+0000 7f707e7fc700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7080100f80 0x7f708019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.710+0000 7f707effd700 1 -- 172.21.15.62:0/4490574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7080101920 msgr2=0x7f70801a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.710+0000 7f707effd700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7080101920 0x7f70801a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.710+0000 7f707effd700 1 -- 172.21.15.62:0/4490574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7080100f80 msgr2=0x7f708019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.710+0000 7f707effd700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7080100f80 0x7f708019e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.710+0000 7f707effd700 1 -- 172.21.15.62:0/4490574 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7078009d00 con 0x7f70800ffd90 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.710+0000 7f707effd700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800ffd90 0x7f708006a010 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f706800d980 tx=0x7f706800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.710+0000 7f7077fff700 1 -- 172.21.15.62:0/4490574 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706800f890 con 0x7f70800ffd90 2024-05-08T22:38:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.710+0000 7f7077fff700 1 -- 172.21.15.62:0/4490574 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7068010460 con 0x7f70800ffd90 2024-05-08T22:38:36.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.710+0000 7f7085567700 1 -- 172.21.15.62:0/4490574 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70801a1210 con 0x7f70800ffd90 2024-05-08T22:38:36.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.710+0000 7f7077fff700 1 -- 172.21.15.62:0/4490574 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70680195c0 con 0x7f70800ffd90 2024-05-08T22:38:36.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.710+0000 7f7085567700 1 -- 172.21.15.62:0/4490574 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70801a1720 con 0x7f70800ffd90 2024-05-08T22:38:36.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.711+0000 7f7085567700 1 -- 172.21.15.62:0/4490574 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7080107140 con 0x7f70800ffd90 2024-05-08T22:38:36.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.712+0000 7f7077fff700 1 -- 172.21.15.62:0/4490574 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f70680105d0 con 0x7f70800ffd90 2024-05-08T22:38:36.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.713+0000 7f7077fff700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f706c0a1120 0x7f706c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:36.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.713+0000 7f707e7fc700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f706c0a1120 0x7f706c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:36.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.713+0000 7f7077fff700 1 -- 172.21.15.62:0/4490574 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8641+0+0 (secure 0 0 0) 0x7f70680cc830 con 0x7f70800ffd90 2024-05-08T22:38:36.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.713+0000 7f707e7fc700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f706c0a1120 0x7f706c0a35e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f708006a5d0 tx=0x7f7070009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:36.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.717+0000 7f7077fff700 1 -- 172.21.15.62:0/4490574 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7068095e20 con 0x7f70800ffd90 2024-05-08T22:38:36.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/2577518990' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:38:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:38:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T22:38:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T22:38:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: Cluster is now healthy 2024-05-08T22:38:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.62:6838/3463716826,v1:172.21.15.62:6841/3463716826] up:boot 2024-05-08T22:38:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.79:6832/120850038,v1:172.21.15.79:6834/120850038] up:boot 2024-05-08T22:38:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.79:6838/2922871166,v1:172.21.15.79:6839/2922871166] up:boot 2024-05-08T22:38:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.66:6834/1223992056,v1:172.21.15.66:6835/1223992056] up:boot 2024-05-08T22:38:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.62:6839/3735792325,v1:172.21.15.62:6840/3735792325] up:boot 2024-05-08T22:38:36.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.79:6833/1327525259,v1:172.21.15.79:6836/1327525259] up:boot 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.79:6835/706019529,v1:172.21.15.79:6837/706019529] up:boot 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.66:6832/846652743,v1:172.21.15.66:6833/846652743] up:boot 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.66:6836/2328033707,v1:172.21.15.66:6837/2328033707] up:boot 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: mds.? [v2:172.21.15.66:6838/2969334547,v1:172.21.15.66:6839/2969334547] up:boot 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: fsmap 12 up:standby 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:38:36.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:38:36.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:36 smithi062 ceph-mon[42571]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:38:36.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:36.927+0000 7f7085567700 1 -- 172.21.15.62:0/4490574 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7080068160 con 0x7f70800ffd90 2024-05-08T22:38:37.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/2577518990' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:38:37.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:38:37.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T22:38:37.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: Cluster is now healthy 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.62:6838/3463716826,v1:172.21.15.62:6841/3463716826] up:boot 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.79:6832/120850038,v1:172.21.15.79:6834/120850038] up:boot 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.79:6838/2922871166,v1:172.21.15.79:6839/2922871166] up:boot 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.66:6834/1223992056,v1:172.21.15.66:6835/1223992056] up:boot 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.62:6839/3735792325,v1:172.21.15.62:6840/3735792325] up:boot 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.79:6833/1327525259,v1:172.21.15.79:6836/1327525259] up:boot 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.79:6835/706019529,v1:172.21.15.79:6837/706019529] up:boot 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.66:6832/846652743,v1:172.21.15.66:6833/846652743] up:boot 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.66:6836/2328033707,v1:172.21.15.66:6837/2328033707] up:boot 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: mds.? [v2:172.21.15.66:6838/2969334547,v1:172.21.15.66:6839/2969334547] up:boot 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: fsmap 12 up:standby 2024-05-08T22:38:37.074 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:38:37.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:38:37.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:38:37.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:38:37.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:38:37.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:38:37.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:38:37.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:38:37.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:38:37.075 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:36 smithi066 ceph-mon[46830]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:38:37.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/2577518990' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:38:37.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T22:38:37.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T22:38:37.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T22:38:37.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: Cluster is now healthy 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.62:6838/3463716826,v1:172.21.15.62:6841/3463716826] up:boot 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.79:6832/120850038,v1:172.21.15.79:6834/120850038] up:boot 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.79:6838/2922871166,v1:172.21.15.79:6839/2922871166] up:boot 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.66:6834/1223992056,v1:172.21.15.66:6835/1223992056] up:boot 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.62:6839/3735792325,v1:172.21.15.62:6840/3735792325] up:boot 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.79:6833/1327525259,v1:172.21.15.79:6836/1327525259] up:boot 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.79:6835/706019529,v1:172.21.15.79:6837/706019529] up:boot 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.66:6832/846652743,v1:172.21.15.66:6833/846652743] up:boot 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.66:6836/2328033707,v1:172.21.15.66:6837/2328033707] up:boot 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: mds.? [v2:172.21.15.66:6838/2969334547,v1:172.21.15.66:6839/2969334547] up:boot 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: fsmap 12 up:standby 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T22:38:37.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T22:38:37.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T22:38:37.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T22:38:37.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T22:38:37.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T22:38:37.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T22:38:37.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T22:38:37.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T22:38:37.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:36 smithi079 ceph-mon[46738]: from='mgr.14150 172.21.15.62:0/2352306819' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T22:38:37.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.789+0000 7f7077fff700 1 -- 172.21.15.62:0/4490574 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) v1 ==== 181+0+0 (secure 0 0 0) 0x7f70680160d0 con 0x7f70800ffd90 2024-05-08T22:38:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 -- 172.21.15.62:0/4490574 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f706c0a1120 msgr2=0x7f706c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f706c0a1120 0x7f706c0a35e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f708006a5d0 tx=0x7f7070009260 comp rx=0 tx=0).stop 2024-05-08T22:38:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 -- 172.21.15.62:0/4490574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800ffd90 msgr2=0x7f708006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800ffd90 0x7f708006a010 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f706800d980 tx=0x7f706800dc90 comp rx=0 tx=0).stop 2024-05-08T22:38:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 -- 172.21.15.62:0/4490574 shutdown_connections 2024-05-08T22:38:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7080101920 0x7f70801a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800ffd90 0x7f708006a010 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f706c0a1120 0x7f706c0a35e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 --2- 172.21.15.62:0/4490574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7080100f80 0x7f708019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 -- 172.21.15.62:0/4490574 >> 172.21.15.62:0/4490574 conn(0x7f70800fb340 msgr2=0x7f7080104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 -- 172.21.15.62:0/4490574 shutdown_connections 2024-05-08T22:38:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.794+0000 7f7085567700 1 -- 172.21.15.62:0/4490574 wait complete. 2024-05-08T22:38:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-05-08T22:38:37.810 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T22:38:37.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.963+0000 7f6d7a041700 1 -- 172.21.15.62:0/2327860820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d74100e70 msgr2=0x7f6d74101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:37.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.963+0000 7f6d7a041700 1 --2- 172.21.15.62:0/2327860820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d74100e70 0x7f6d74101290 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009990 tx=0x7f6d64009ca0 comp rx=0 tx=0).stop 2024-05-08T22:38:37.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.963+0000 7f6d7a041700 1 -- 172.21.15.62:0/2327860820 shutdown_connections 2024-05-08T22:38:37.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.963+0000 7f6d7a041700 1 --2- 172.21.15.62:0/2327860820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74102980 0x7f6d741092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:37.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.963+0000 7f6d7a041700 1 --2- 172.21.15.62:0/2327860820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d74100e70 0x7f6d74101290 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:37.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.963+0000 7f6d7a041700 1 --2- 172.21.15.62:0/2327860820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d74101fc0 0x7f6d74102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:37.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.963+0000 7f6d7a041700 1 -- 172.21.15.62:0/2327860820 >> 172.21.15.62:0/2327860820 conn(0x7f6d740fc400 msgr2=0x7f6d740fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:37.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.963+0000 7f6d7a041700 1 -- 172.21.15.62:0/2327860820 shutdown_connections 2024-05-08T22:38:37.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.963+0000 7f6d7a041700 1 -- 172.21.15.62:0/2327860820 wait complete. 2024-05-08T22:38:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.964+0000 7f6d7a041700 1 Processor -- start 2024-05-08T22:38:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.964+0000 7f6d7a041700 1 -- start start 2024-05-08T22:38:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.964+0000 7f6d7a041700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74100e70 0x7f6d74196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.964+0000 7f6d7a041700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d74101fc0 0x7f6d74197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.964+0000 7f6d7a041700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d74102980 0x7f6d7419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.964+0000 7f6d7a041700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7419e950 con 0x7f6d74101fc0 2024-05-08T22:38:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.964+0000 7f6d7a041700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7419ea90 con 0x7f6d74100e70 2024-05-08T22:38:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.964+0000 7f6d7a041700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7419ec20 con 0x7f6d74102980 2024-05-08T22:38:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.964+0000 7f6d72ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d74101fc0 0x7f6d74197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:37.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d72ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d74101fc0 0x7f6d74197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38728/0 (socket says 172.21.15.62:38728) 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d72ffd700 1 -- 172.21.15.62:0/3408517009 learned_addr learned my addr 172.21.15.62:0/3408517009 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d73fff700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d74102980 0x7f6d7419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d737fe700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74100e70 0x7f6d74196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d72ffd700 1 -- 172.21.15.62:0/3408517009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d74102980 msgr2=0x7f6d7419e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d72ffd700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d74102980 0x7f6d7419e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d72ffd700 1 -- 172.21.15.62:0/3408517009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74100e70 msgr2=0x7f6d74196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d72ffd700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74100e70 0x7f6d74196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d72ffd700 1 -- 172.21.15.62:0/3408517009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640095f0 con 0x7f6d74101fc0 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d72ffd700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d74101fc0 0x7f6d74197360 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00d950 tx=0x7f6d5c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d70ff9700 1 -- 172.21.15.62:0/3408517009 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c009940 con 0x7f6d74101fc0 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d70ff9700 1 -- 172.21.15.62:0/3408517009 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d5c010460 con 0x7f6d74101fc0 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d70ff9700 1 -- 172.21.15.62:0/3408517009 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00f620 con 0x7f6d74101fc0 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d7a041700 1 -- 172.21.15.62:0/3408517009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7419ef00 con 0x7f6d74101fc0 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.965+0000 7f6d7a041700 1 -- 172.21.15.62:0/3408517009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7419f320 con 0x7f6d74101fc0 2024-05-08T22:38:37.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.966+0000 7f6d7a041700 1 -- 172.21.15.62:0/3408517009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d740674f0 con 0x7f6d74101fc0 2024-05-08T22:38:37.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.967+0000 7f6d70ff9700 1 -- 172.21.15.62:0/3408517009 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6d5c0108d0 con 0x7f6d74101fc0 2024-05-08T22:38:37.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.968+0000 7f6d70ff9700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d600a1120 0x7f6d600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T22:38:37.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.968+0000 7f6d70ff9700 1 -- 172.21.15.62:0/3408517009 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8087+0+0 (secure 0 0 0) 0x7f6d5c0cc710 con 0x7f6d74101fc0 2024-05-08T22:38:37.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.968+0000 7f6d737fe700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d600a1120 0x7f6d600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T22:38:37.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.968+0000 7f6d737fe700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d600a1120 0x7f6d600a35e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6d64006010 tx=0x7f6d640037b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T22:38:37.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:37.972+0000 7f6d70ff9700 1 -- 172.21.15.62:0/3408517009 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6d740674f0 con 0x7f6d74101fc0 2024-05-08T22:38:38.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:37 smithi066 ceph-mon[46830]: pgmap v328: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-08T22:38:38.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:37 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/4490574' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:38:38.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:37 smithi079 ceph-mon[46738]: pgmap v328: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-08T22:38:38.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:37 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/4490574' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:38:38.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:37 smithi062 ceph-mon[42571]: pgmap v328: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-08T22:38:38.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:37 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/4490574' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:38:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.188+0000 7f6d7a041700 1 -- 172.21.15.62:0/3408517009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6d74067480 con 0x7f6d74101fc0 2024-05-08T22:38:38.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.799+0000 7f6d70ff9700 1 -- 172.21.15.62:0/3408517009 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6d5c099970 con 0x7f6d74101fc0 2024-05-08T22:38:38.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 -- 172.21.15.62:0/3408517009 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d600a1120 msgr2=0x7f6d600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d600a1120 0x7f6d600a35e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6d64006010 tx=0x7f6d640037b0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 -- 172.21.15.62:0/3408517009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d74101fc0 msgr2=0x7f6d74197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T22:38:38.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d74101fc0 0x7f6d74197360 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00d950 tx=0x7f6d5c00dc60 comp rx=0 tx=0).stop 2024-05-08T22:38:38.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 -- 172.21.15.62:0/3408517009 shutdown_connections 2024-05-08T22:38:38.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d74102980 0x7f6d7419e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.62:6800/4071146880,v1:172.21.15.62:6801/4071146880] conn(0x7f6d600a1120 0x7f6d600a35e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74100e70 0x7f6d74196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 --2- 172.21.15.62:0/3408517009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d74101fc0 0x7f6d74197360 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T22:38:38.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 -- 172.21.15.62:0/3408517009 >> 172.21.15.62:0/3408517009 conn(0x7f6d740fc400 msgr2=0x7f6d74105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T22:38:38.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 -- 172.21.15.62:0/3408517009 shutdown_connections 2024-05-08T22:38:38.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-05-08T22:38:38.802+0000 7f6d7a041700 1 -- 172.21.15.62:0/3408517009 wait complete. 2024-05-08T22:38:38.803 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-05-08T22:38:38.818 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:38:38.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:38:38.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:38:38.943 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-08T22:38:38.943 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T22:38:38.962 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T22:38:38.982 DEBUG:teuthology.orchestra.run.smithi079:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T22:38:39.002 INFO:tasks.cephadm:Stopping all daemons... 2024-05-08T22:38:39.003 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-08T22:38:39.003 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.a 2024-05-08T22:38:39.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:38 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/4490574' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:38:39.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:38 smithi066 ceph-mon[46830]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T22:38:39.073 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:38 smithi066 ceph-mon[46830]: from='client.? 172.21.15.62:0/3408517009' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:38:39.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:38 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/4490574' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:38:39.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:38 smithi079 ceph-mon[46738]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T22:38:39.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:38 smithi079 ceph-mon[46738]: from='client.? 172.21.15.62:0/3408517009' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:38:39.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:38 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/4490574' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T22:38:39.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:38 smithi062 ceph-mon[42571]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T22:38:39.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:38 smithi062 ceph-mon[42571]: from='client.? 172.21.15.62:0/3408517009' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T22:38:39.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:39 smithi062 systemd[1]: Stopping Ceph mon.a for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:38:39.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:39 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-a[42548]: 2024-05-08T22:38:39.248+0000 7f1e88154700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:38:39.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:39 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-a[42548]: 2024-05-08T22:38:39.248+0000 7f1e88154700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-08T22:38:39.672 INFO:journalctl@ceph.mon.a.smithi062.stdout:May 08 22:38:39 smithi062 bash[112366]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-a 2024-05-08T22:38:39.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.a.service' 2024-05-08T22:38:39.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:38:39.828 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-08T22:38:39.828 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-08T22:38:39.828 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.b 2024-05-08T22:38:40.298 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:39 smithi066 systemd[1]: Stopping Ceph mon.b for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:38:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:40 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-b[46808]: 2024-05-08T22:38:40.068+0000 7f3d2101d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:38:40.299 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:40 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-b[46808]: 2024-05-08T22:38:40.068+0000 7f3d2101d700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-08T22:38:40.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 08 22:38:40 smithi066 bash[86643]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-b 2024-05-08T22:38:40.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.b.service' 2024-05-08T22:38:40.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:38:40.629 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-08T22:38:40.629 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-08T22:38:40.629 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.c 2024-05-08T22:38:41.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:40 smithi079 systemd[1]: Stopping Ceph mon.c for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:38:41.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:40 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-c[46713]: 2024-05-08T22:38:40.872+0000 7f4f6bcfb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:38:41.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:40 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-c[46713]: 2024-05-08T22:38:40.872+0000 7f4f6bcfb700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-08T22:38:41.562 INFO:journalctl@ceph.mon.c.smithi079.stdout:May 08 22:38:41 smithi079 bash[88203]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mon-c 2024-05-08T22:38:41.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mon.c.service' 2024-05-08T22:38:41.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:38:41.620 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-08T22:38:41.620 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-08T22:38:41.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.x 2024-05-08T22:38:41.921 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:38:41 smithi062 systemd[1]: Stopping Ceph mgr.x for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:38:41.921 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:38:41 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x[42823]: 2024-05-08T22:38:41.864+0000 7f67fda0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:38:42.250 INFO:journalctl@ceph.mgr.x.smithi062.stdout:May 08 22:38:42 smithi062 bash[112504]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-x 2024-05-08T22:38:42.257 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-08T22:38:42.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:38:42.295 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-08T22:38:42.296 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-08T22:38:42.296 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.y 2024-05-08T22:38:42.648 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:38:42 smithi066 systemd[1]: Stopping Ceph mgr.y for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:38:42.923 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:38:42 smithi066 bash[86795]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-y 2024-05-08T22:38:42.923 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 08 22:38:42 smithi066 systemd[1]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-08T22:38:42.929 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-08T22:38:42.967 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:38:42.967 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-08T22:38:42.968 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-08T22:38:42.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.z 2024-05-08T22:38:43.351 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:38:43 smithi079 systemd[1]: Stopping Ceph mgr.z for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:38:43.638 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:38:43 smithi079 bash[88355]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mgr-z 2024-05-08T22:38:43.638 INFO:journalctl@ceph.mgr.z.smithi079.stdout:May 08 22:38:43 smithi079 systemd[1]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-08T22:38:43.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-08T22:38:43.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:38:43.759 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-08T22:38:43.759 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-08T22:38:43.760 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.0 2024-05-08T22:38:44.171 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:38:43 smithi062 systemd[1]: Stopping Ceph osd.0 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:38:44.172 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:38:44 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-0[61728]: 2024-05-08T22:38:44.007+0000 7ff0aa3e4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:38:44.172 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:38:44 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-0[61728]: 2024-05-08T22:38:44.007+0000 7ff0aa3e4700 -1 osd.0 84 *** Got signal Terminated *** 2024-05-08T22:38:44.172 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:38:44 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-0[61728]: 2024-05-08T22:38:44.007+0000 7ff0aa3e4700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:38:49.922 INFO:journalctl@ceph.osd.0.smithi062.stdout:May 08 22:38:49 smithi062 bash[112644]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-0 2024-05-08T22:38:50.954 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.0.service' 2024-05-08T22:38:50.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:38:50.999 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-08T22:38:50.999 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-08T22:38:50.999 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.1 2024-05-08T22:38:51.323 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:38:51 smithi066 systemd[1]: Stopping Ceph osd.1 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:38:51.323 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:38:51 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-1[51799]: 2024-05-08T22:38:51.242+0000 7ff0af71f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:38:51.323 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:38:51 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-1[51799]: 2024-05-08T22:38:51.242+0000 7ff0af71f700 -1 osd.1 84 *** Got signal Terminated *** 2024-05-08T22:38:51.323 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:38:51 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-1[51799]: 2024-05-08T22:38:51.242+0000 7ff0af71f700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:38:57.073 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 08 22:38:56 smithi066 bash[86934]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-1 2024-05-08T22:38:57.956 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.1.service' 2024-05-08T22:38:58.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:38:58.012 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-08T22:38:58.012 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-08T22:38:58.012 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.2 2024-05-08T22:38:58.388 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:38:58 smithi079 systemd[1]: Stopping Ceph osd.2 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:38:58.388 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:38:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-2[52532]: 2024-05-08T22:38:58.269+0000 7fe7fc5a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:38:58.388 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:38:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-2[52532]: 2024-05-08T22:38:58.269+0000 7fe7fc5a2700 -1 osd.2 84 *** Got signal Terminated *** 2024-05-08T22:38:58.389 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:38:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-2[52532]: 2024-05-08T22:38:58.269+0000 7fe7fc5a2700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:39:04.388 INFO:journalctl@ceph.osd.2.smithi079.stdout:May 08 22:39:03 smithi079 bash[88508]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-2 2024-05-08T22:39:05.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.2.service' 2024-05-08T22:39:05.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:39:05.252 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-08T22:39:05.252 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-08T22:39:05.253 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.3 2024-05-08T22:39:05.671 INFO:journalctl@ceph.osd.3.smithi062.stdout:May 08 22:39:05 smithi062 systemd[1]: Stopping Ceph osd.3 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:39:05.672 INFO:journalctl@ceph.osd.3.smithi062.stdout:May 08 22:39:05 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-3[70463]: 2024-05-08T22:39:05.503+0000 7f5ede571700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:39:05.672 INFO:journalctl@ceph.osd.3.smithi062.stdout:May 08 22:39:05 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-3[70463]: 2024-05-08T22:39:05.503+0000 7f5ede571700 -1 osd.3 84 *** Got signal Terminated *** 2024-05-08T22:39:05.672 INFO:journalctl@ceph.osd.3.smithi062.stdout:May 08 22:39:05 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-3[70463]: 2024-05-08T22:39:05.503+0000 7f5ede571700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:39:10.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:10 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:10.277+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:11.421 INFO:journalctl@ceph.osd.3.smithi062.stdout:May 08 22:39:11 smithi062 bash[112916]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-3 2024-05-08T22:39:11.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:11 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:11.261+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:12.449 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.3.service' 2024-05-08T22:39:12.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:39:12.491 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-08T22:39:12.491 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-08T22:39:12.491 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.4 2024-05-08T22:39:12.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:12 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:12.305+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:12.823 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 08 22:39:12 smithi066 systemd[1]: Stopping Ceph osd.4 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:39:12.823 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 08 22:39:12 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-4[59782]: 2024-05-08T22:39:12.739+0000 7f2edd7d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:39:12.824 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 08 22:39:12 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-4[59782]: 2024-05-08T22:39:12.739+0000 7f2edd7d2700 -1 osd.4 84 *** Got signal Terminated *** 2024-05-08T22:39:12.824 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 08 22:39:12 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-4[59782]: 2024-05-08T22:39:12.739+0000 7f2edd7d2700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:39:13.390 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:13 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:13.325+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:13.391 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:13 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:13.127+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:13.421 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:12 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:12.986+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:13.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:13 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:13.509+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:14.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:13 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:13.966+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:14.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:13 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:13.799+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:14.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:14 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:14.359+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:14.638 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:14 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:14.150+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:14.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:14 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:14.521+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:15.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:14 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:14.794+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:15.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:14 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:14.959+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:15.388 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:15 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:15.102+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:15.388 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:15 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:15.330+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:15.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:15 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:15.562+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:16.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:15 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:15.769+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:16.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:15 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:15.926+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:16.388 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:16 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:16.310+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:16.388 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:16 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:16.081+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:16.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:16 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:16.532+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:17.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:16 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:16.786+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:17.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:16 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:16.901+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:17.388 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:17 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:17.315+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:17.389 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:17 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:17.103+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:17.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:17 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:17.559+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:18.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:17 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:17.770+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:18.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:17 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:17.877+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:18.388 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:18 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:18.074+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:18.388 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:18 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:18.302+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:18.389 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:18 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:18.302+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:18.573 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 08 22:39:18 smithi066 bash[87203]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-4 2024-05-08T22:39:18.828 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:18 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:18.577+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:18.829 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:18 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:18.577+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:19.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:18 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:18.768+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:19.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:18 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:18.885+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:19.279 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:19 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:19.053+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:19.280 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:19 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:19.010+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:19.494 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.4.service' 2024-05-08T22:39:19.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:39:19.537 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-08T22:39:19.537 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-08T22:39:19.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.5 2024-05-08T22:39:19.566 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:19 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:19.281+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:19.567 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:19 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:19.281+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:19.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:19 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:19.471+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:19.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:19 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:19.613+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:19.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:19 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:19.613+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:19.888 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:19 smithi079 systemd[1]: Stopping Ceph osd.5 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:39:19.888 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:19 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:19.805+0000 7fce82cf2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:39:19.888 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:19 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:19.805+0000 7fce82cf2700 -1 osd.5 84 *** Got signal Terminated *** 2024-05-08T22:39:19.888 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:19 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:19.805+0000 7fce82cf2700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:39:20.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:19 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:19.818+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:20.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:19 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:19.818+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:20.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:19 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:19.927+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:20.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:19 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:19.927+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:20.388 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:20 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:20.063+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:20.388 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:20 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:20.063+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.770880+0000 front 2024-05-08T22:38:55.770913+0000 (oldest deadline 2024-05-08T22:39:19.270891+0000) 2024-05-08T22:39:20.389 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:20 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:20.022+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:20.389 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:20 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:20.245+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:20.389 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:20 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:20.245+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:20.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:20 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:20.489+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:20.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:20 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:20.616+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:20.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:20 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:20.616+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:21.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:20 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:20.861+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:21.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:20 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:20.861+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:21.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:20 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:20.902+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:21.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:20 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:20.902+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:21.388 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:21 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:21.015+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:21.388 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:21 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:21.246+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:21.388 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:21 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:21.246+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:21.389 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:21 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:21.040+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:21.389 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:21 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:21.040+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.770880+0000 front 2024-05-08T22:38:55.770913+0000 (oldest deadline 2024-05-08T22:39:19.270891+0000) 2024-05-08T22:39:21.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:21 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:21.493+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:21.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:21 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:21.570+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:21.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:21 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:21.570+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:22.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:21 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:21.891+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:22.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:21 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:21.891+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:22.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:21 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:21.853+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:22.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:21 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:21.853+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:22.275 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:21 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:21.991+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:22.275 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:21 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:21.991+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.770880+0000 front 2024-05-08T22:38:55.770913+0000 (oldest deadline 2024-05-08T22:39:19.270891+0000) 2024-05-08T22:39:22.275 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:22 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:22.031+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:22.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:22 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:22.274+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:22.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:22 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:22.274+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:22.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:22 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:22.469+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:22.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:22 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:22.536+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:22.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:22 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:22.536+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:23.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:22 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:22.868+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:23.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:22 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:22.868+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:23.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:22 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:22.852+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:23.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:22 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:22.852+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:23.285 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:23 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:23.062+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:23.286 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:23 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:23.025+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:23.286 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:23 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:23.025+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.770880+0000 front 2024-05-08T22:38:55.770913+0000 (oldest deadline 2024-05-08T22:39:19.270891+0000) 2024-05-08T22:39:23.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:23 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:23.285+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:23.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:23 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:23.285+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:23.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:23 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:23.518+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:23.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:23 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:23.543+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:23.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:23 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:23.543+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:24.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:23 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:23.866+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:24.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:23 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:23.866+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:24.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:23 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:23.870+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:24.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:23 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:23.870+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:24.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:23 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:23.870+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:24.388 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:24 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:24.052+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:24.388 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:24 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:24.256+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:24.388 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:24 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:24.256+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:24.389 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:24 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:24.049+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.869135+0000 front 2024-05-08T22:38:48.869072+0000 (oldest deadline 2024-05-08T22:39:12.269788+0000) 2024-05-08T22:39:24.389 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:24 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:24.049+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.770880+0000 front 2024-05-08T22:38:55.770913+0000 (oldest deadline 2024-05-08T22:39:19.270891+0000) 2024-05-08T22:39:24.389 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:24 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5[61132]: 2024-05-08T22:39:24.049+0000 7fce7fae9700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.271233+0000 front 2024-05-08T22:38:59.271257+0000 (oldest deadline 2024-05-08T22:39:23.971401+0000) 2024-05-08T22:39:24.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:24 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:24.515+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:24.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:24 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:24.571+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:24.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:24 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:24.571+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:25.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:24 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:24.846+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:25.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:24 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:24.846+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:25.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:24 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:24.842+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:25.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:24 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:24.842+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:25.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:24 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:24.842+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:25.292 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:25 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:25.004+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:25.292 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:25 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:25.004+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:25.638 INFO:journalctl@ceph.osd.5.smithi079.stdout:May 08 22:39:25 smithi079 bash[88766]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-5 2024-05-08T22:39:25.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:25 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:25.290+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:25.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:25 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:25.290+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:25.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:25 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:25.563+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:25.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:25 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:25.561+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:25.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:25 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:25.561+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:26.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:26 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:26.001+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:26.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:26 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:26.001+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:26.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:25 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:25.893+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:26.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:25 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:25.893+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:26.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:25 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:25.893+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.854581+0000 front 2024-05-08T22:39:02.854430+0000 (oldest deadline 2024-05-08T22:39:25.154291+0000) 2024-05-08T22:39:26.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:25 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:25.866+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:26.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:25 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:25.866+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:26.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:25 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:25.866+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:26.559 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:26 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:26.301+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:26.559 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:26 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:26.301+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:26.567 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.5.service' 2024-05-08T22:39:26.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:39:26.650 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-08T22:39:26.650 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-08T22:39:26.650 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.6 2024-05-08T22:39:27.073 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:26 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:26.600+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:27.073 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:26 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:26.600+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:27.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:26 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:26.609+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:27.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:26 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:26.609+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:27.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:26.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:27.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:26.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:27.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:26.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:27.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:26 smithi062 systemd[1]: Stopping Ceph osd.6 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:39:27.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:26.900+0000 7f7e9faaf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:39:27.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:26.900+0000 7f7e9faaf700 -1 osd.6 84 *** Got signal Terminated *** 2024-05-08T22:39:27.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:26.900+0000 7f7e9faaf700 -1 osd.6 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:39:27.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:26.922+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:27.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:26.922+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:27.173 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:26 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:26.922+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.854581+0000 front 2024-05-08T22:39:02.854430+0000 (oldest deadline 2024-05-08T22:39:25.154291+0000) 2024-05-08T22:39:27.284 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:26 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:26.969+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:27.284 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:26 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:26.969+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:27.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:27 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:27.284+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:27.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:27 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:27.284+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:28.073 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:27 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:27.587+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:28.073 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:27 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:27.587+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:28.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:27 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:27.587+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:28.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:27 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:27.587+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:28.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:27 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:27.933+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:28.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:27 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:27.933+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:28.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:27 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:27.933+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.854581+0000 front 2024-05-08T22:39:02.854430+0000 (oldest deadline 2024-05-08T22:39:25.154291+0000) 2024-05-08T22:39:28.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:27 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:27.827+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:28.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:27 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:27.827+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:28.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:27 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:27.827+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:28.269 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:27 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:27.939+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:28.269 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:27 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:27.939+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:28.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:28 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:28.268+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:28.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:28 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:28.268+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:28.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:28 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:28.268+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:28.824 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:28 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:28.570+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:28.824 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:28 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:28.570+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:28.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:28 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:28.612+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:28.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:28 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:28.612+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:28.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:28 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:28.612+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:29.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:28 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:28.942+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:29.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:28 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:28.942+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:29.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:28 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:28.942+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.854581+0000 front 2024-05-08T22:39:02.854430+0000 (oldest deadline 2024-05-08T22:39:25.154291+0000) 2024-05-08T22:39:29.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:28 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:28.825+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:29.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:28 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:28.825+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:29.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:28 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:28.825+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:29.298 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:28 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:28.924+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:29.298 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:28 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:28.924+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:29.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:29 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:29.298+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:29.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:29 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:29.298+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:29.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:29 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:29.298+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:29.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:29 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:29.521+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:29.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:29 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:29.521+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:29.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:29 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:29.658+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:29.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:29 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:29.658+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:29.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:29 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:29.658+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:30.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:29 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:29.988+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:30.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:29 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:29.988+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:30.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:29 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:29.988+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.854581+0000 front 2024-05-08T22:39:02.854430+0000 (oldest deadline 2024-05-08T22:39:25.154291+0000) 2024-05-08T22:39:30.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:29 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:29.801+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:30.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:29 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:29.801+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:30.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:29 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:29.801+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:30.269 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:29 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:29.962+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:30.269 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:29 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:29.962+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:30.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:30 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:30.269+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:30.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:30 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:30.269+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:30.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:30 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:30.269+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:30.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:30 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:30.537+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:30.824 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:30 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:30.537+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:30.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:30 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:30.704+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:30.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:30 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:30.704+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:30.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:30 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:30.704+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:31.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:31 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:31.014+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.252015+0000 front 2024-05-08T22:38:48.252106+0000 (oldest deadline 2024-05-08T22:39:12.952246+0000) 2024-05-08T22:39:31.171 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:31 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:31.014+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.952662+0000 front 2024-05-08T22:38:52.952713+0000 (oldest deadline 2024-05-08T22:39:18.852874+0000) 2024-05-08T22:39:31.172 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:31 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6[79238]: 2024-05-08T22:39:31.014+0000 7f7e9c8a6700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.854581+0000 front 2024-05-08T22:39:02.854430+0000 (oldest deadline 2024-05-08T22:39:25.154291+0000) 2024-05-08T22:39:31.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:30 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:30.777+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:31.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:30 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:30.777+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:31.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:30 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:30.777+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:31.227 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:30 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:30.927+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:31.227 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:30 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:30.927+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:31.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:31 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:31.227+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:31.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:31 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:31.227+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:31.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:31 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:31.227+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:31.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:31 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:31.492+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:31.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:31 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:31.492+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:31.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:31 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:31.680+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:31.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:31 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:31.680+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:31.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:31 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:31.680+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:32.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:31 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:31.887+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:32.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:31 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:31.887+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:32.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:31 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:31.809+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:32.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:31 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:31.809+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:32.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:31 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:31.809+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:32.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:32 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:32.273+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:32.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:32 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:32.273+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:32.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:32 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:32.273+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:32.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:32 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:32.459+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:32.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:32 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:32.459+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:32.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:32 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:32.653+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:32.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:32 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:32.653+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:32.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:32 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:32.653+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:32.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:32 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:32.653+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:32.838 INFO:journalctl@ceph.osd.6.smithi062.stdout:May 08 22:39:32 smithi062 bash[113172]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-6 2024-05-08T22:39:33.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:32 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:32.848+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:33.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:32 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:32.848+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:33.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:32 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:32.838+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:33.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:32 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:32.838+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:33.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:32 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:32.838+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:33.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:33 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:33.319+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:33.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:33 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:33.319+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:33.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:33 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:33.319+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:33.720 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.6.service' 2024-05-08T22:39:33.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:39:33.762 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-08T22:39:33.762 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-08T22:39:33.762 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.7 2024-05-08T22:39:33.793 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:33 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:33.456+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:33.793 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:33 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:33.456+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:33.793 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:33 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:33.456+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.977554+0000 front 2024-05-08T22:39:08.977669+0000 (oldest deadline 2024-05-08T22:39:33.077635+0000) 2024-05-08T22:39:33.793 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:33 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:33.669+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:33.793 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:33 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:33.669+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:33.793 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:33 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:33.669+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:33.794 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:33 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:33.669+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:34.073 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:33 smithi066 systemd[1]: Stopping Ceph osd.7 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:39:34.073 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:34 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:34.004+0000 7fa1d33dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:39:34.073 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:34 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:34.004+0000 7fa1d33dd700 -1 osd.7 84 *** Got signal Terminated *** 2024-05-08T22:39:34.074 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:34 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:34.004+0000 7fa1d33dd700 -1 osd.7 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:39:34.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:33 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:33.818+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:34.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:33 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:33.818+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:34.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:33 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:33.818+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:34.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:33 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:33.805+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:34.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:33 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:33.805+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:34.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:33 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:33.805+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:34.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:33 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:33.805+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:34.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:34 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:34.311+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:34.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:34 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:34.311+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:34.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:34 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:34.311+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:34.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:34 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:34.482+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:34.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:34 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:34.482+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:34.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:34 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:34.482+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.977554+0000 front 2024-05-08T22:39:08.977669+0000 (oldest deadline 2024-05-08T22:39:33.077635+0000) 2024-05-08T22:39:34.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:34 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:34.641+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:34.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:34 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:34.641+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:34.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:34 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:34.641+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:34.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:34 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:34.641+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:35.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:34 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:34.847+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:35.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:34 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:34.847+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:35.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:34 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:34.847+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:35.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:34 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:34.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:35.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:34 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:34.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:35.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:34 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:34.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:35.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:34 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:34.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:35.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:35.308+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:35.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:35.308+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:35.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:35.308+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:35.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:35.518+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:35.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:35.518+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:35.824 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:35.518+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.977554+0000 front 2024-05-08T22:39:08.977669+0000 (oldest deadline 2024-05-08T22:39:33.077635+0000) 2024-05-08T22:39:35.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:35.627+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:35.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:35.627+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:35.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:35.627+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:35.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:35 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:35.627+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:36.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:35.868+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:36.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:35.868+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:36.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:35 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:35.868+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:36.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:35 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:35.774+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:36.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:35 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:35.774+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:36.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:35 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:35.774+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:36.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:35 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:35.774+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:36.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:36 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:36.354+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:36.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:36 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:36.354+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:36.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:36 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:36.354+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:36.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:36 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:36.539+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:36.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:36 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:36.539+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:36.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:36 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:36.539+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.977554+0000 front 2024-05-08T22:39:08.977669+0000 (oldest deadline 2024-05-08T22:39:33.077635+0000) 2024-05-08T22:39:36.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:36 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:36.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:36.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:36 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:36.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:36.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:36 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:36.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:36.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:36 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:36.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:37.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:36 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:36.831+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:37.139 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:36 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:36.831+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:37.139 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:36 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:36.831+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:37.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:36 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:36.745+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:37.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:36 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:36.745+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:37.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:36 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:36.745+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:37.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:36 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:36.745+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:37.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:37 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:37.379+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:37.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:37 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:37.379+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:37.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:37 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:37.379+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:37.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:37 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:37.511+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:37.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:37 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:37.511+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:37.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:37 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:37.511+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.977554+0000 front 2024-05-08T22:39:08.977669+0000 (oldest deadline 2024-05-08T22:39:33.077635+0000) 2024-05-08T22:39:37.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:37 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:37.657+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:37.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:37 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:37.657+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:37.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:37 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:37.657+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:37.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:37 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:37.657+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:38.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:37 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:37.858+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:38.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:37 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:37.858+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:38.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:37 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:37.858+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:38.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:37 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:37.751+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:38.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:37 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:37.751+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:38.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:37 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:37.751+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:38.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:37 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:37.751+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:38.818 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:38 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:38.411+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:38.819 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:38 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:38.411+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:38.819 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:38 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:38.411+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:38.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:38 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:38.497+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:54.574133+0000 front 2024-05-08T22:38:54.574225+0000 (oldest deadline 2024-05-08T22:39:18.674315+0000) 2024-05-08T22:39:38.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:38 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:38.497+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:38:59.775567+0000 front 2024-05-08T22:38:59.775708+0000 (oldest deadline 2024-05-08T22:39:25.675439+0000) 2024-05-08T22:39:38.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:38 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:38.497+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.977554+0000 front 2024-05-08T22:39:08.977669+0000 (oldest deadline 2024-05-08T22:39:33.077635+0000) 2024-05-08T22:39:38.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:38 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7[67972]: 2024-05-08T22:39:38.497+0000 7fa1cc1cc700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:13.077913+0000 front 2024-05-08T22:39:13.077930+0000 (oldest deadline 2024-05-08T22:39:38.378201+0000) 2024-05-08T22:39:38.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:38 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:38.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:38.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:38 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:38.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:38.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:38 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:38.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:38.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:38 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:38.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:39.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:38 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:38.819+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:39.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:38 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:38.819+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:39.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:38 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:38.819+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:39.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:38 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:38.782+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:39.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:38 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:38.782+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:39.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:38 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:38.782+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:39.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:38 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:38.782+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:39.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:39 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:39.380+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:39.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:39 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:39.380+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:39.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:39 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:39.380+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:39.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:39 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:39.380+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:39.823 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 08 22:39:39 smithi066 bash[87458]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-7 2024-05-08T22:39:39.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:39 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:39.613+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:39.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:39 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:39.613+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:39.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:39 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:39.613+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:39.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:39 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:39.613+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:40.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:39 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:39.802+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:40.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:39 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:39.802+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:40.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:39 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:39.802+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:40.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:39 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:39.751+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:40.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:39 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:39.751+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:40.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:39 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:39.751+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:40.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:39 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:39.751+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:40.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:39 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:39.751+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:40.716 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:40 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:40.587+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:40.716 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:40 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:40.587+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:40.716 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:40 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:40.587+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:40.716 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:40 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:40.587+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:40.723 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.7.service' 2024-05-08T22:39:40.802 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:40 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:40.402+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:40.802 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:40 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:40.402+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:40.802 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:40 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:40.402+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:40.803 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:40 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:40.402+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:40.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:39:40.808 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-08T22:39:40.808 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-08T22:39:40.808 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.8 2024-05-08T22:39:41.059 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:40 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:40.803+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:41.059 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:40 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:40.803+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:41.059 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:40 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:40.803+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:41.059 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:40 smithi079 systemd[1]: Stopping Ceph osd.8 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:39:41.059 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:41 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:41.059+0000 7f0954c4b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:39:41.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:40 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:40.737+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:41.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:40 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:40.737+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:41.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:40 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:40.737+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:41.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:40 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:40.737+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:41.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:40 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:40.737+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:41.384 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:41 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:41.059+0000 7f0954c4b700 -1 osd.8 84 *** Got signal Terminated *** 2024-05-08T22:39:41.384 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:41 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:41.059+0000 7f0954c4b700 -1 osd.8 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:39:41.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:41 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:41.384+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:41.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:41 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:41.384+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:41.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:41 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:41.384+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:41.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:41 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:41.384+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:41.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:41 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:41.555+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:41.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:41 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:41.555+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:41.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:41 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:41.555+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:41.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:41 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:41.555+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:42.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:41 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:41.825+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:42.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:41 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:41.825+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:42.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:41 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:41.825+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:42.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:41 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:41.825+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.680428+0000 front 2024-05-08T22:39:17.680475+0000 (oldest deadline 2024-05-08T22:39:41.780642+0000) 2024-05-08T22:39:42.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:41 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:41.786+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:42.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:41 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:41.786+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:42.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:41 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:41.786+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:42.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:41 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:41.786+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:42.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:41 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:41.786+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:42.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:42 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:42.549+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:42.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:42 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:42.549+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:42.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:42 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:42.549+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:42.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:42 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:42.549+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:42.833 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:42 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:42.405+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:42.833 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:42 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:42.405+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:42.833 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:42 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:42.405+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:42.833 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:42 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:42.405+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:43.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:42 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:42.833+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:43.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:42 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:42.833+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:43.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:42 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:42.833+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:43.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:42 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:42.833+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.680428+0000 front 2024-05-08T22:39:17.680475+0000 (oldest deadline 2024-05-08T22:39:41.780642+0000) 2024-05-08T22:39:43.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:42 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:42.815+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:43.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:42 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:42.815+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:43.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:42 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:42.815+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:43.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:42 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:42.815+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:43.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:42 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:42.815+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:43.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:43 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:43.386+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:43.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:43 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:43.386+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:43.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:43 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:43.386+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:43.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:43 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:43.386+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:43.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:43 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:43.529+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:43.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:43 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:43.529+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:43.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:43 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:43.529+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:43.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:43 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:43.529+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:43.824 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:43 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:43.529+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:44.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:43 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:43.857+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:44.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:43 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:43.857+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:44.139 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:43 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:43.857+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:44.139 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:43 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:43.857+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.680428+0000 front 2024-05-08T22:39:17.680475+0000 (oldest deadline 2024-05-08T22:39:41.780642+0000) 2024-05-08T22:39:44.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:43 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:43.817+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:44.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:43 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:43.817+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:44.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:43 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:43.817+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:44.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:43 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:43.817+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:44.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:43 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:43.817+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:44.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:44 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:44.387+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:44.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:44 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:44.387+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:44.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:44 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:44.387+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:44.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:44 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:44.387+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:45.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:44 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:44.576+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:45.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:44 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:44.576+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:45.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:44 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:44.576+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:45.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:44 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:44.576+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:45.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:44 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:44.576+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:45.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:44 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:44.844+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:45.139 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:44 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:44.844+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:45.139 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:44 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:44.844+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:45.139 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:44 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:44.844+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.680428+0000 front 2024-05-08T22:39:17.680475+0000 (oldest deadline 2024-05-08T22:39:41.780642+0000) 2024-05-08T22:39:45.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:44 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:44.813+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:45.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:44 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:44.813+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:45.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:44 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:44.813+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:45.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:44 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:44.813+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:45.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:44 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:44.813+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:45.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:45 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:45.343+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:45.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:45 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:45.343+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:45.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:45 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:45.343+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:45.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:45 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:45.343+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:45.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:45 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:45.343+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:46.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:45 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:45.584+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:46.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:45 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:45.584+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:46.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:45 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:45.584+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:46.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:45 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:45.584+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:46.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:45 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:45.584+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:46.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:45 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:45.806+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.177139+0000 front 2024-05-08T22:38:53.177073+0000 (oldest deadline 2024-05-08T22:39:18.477120+0000) 2024-05-08T22:39:46.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:45 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:45.806+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:01.278384+0000 front 2024-05-08T22:39:01.278487+0000 (oldest deadline 2024-05-08T22:39:24.778487+0000) 2024-05-08T22:39:46.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:45 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:45.806+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:07.079529+0000 front 2024-05-08T22:39:07.079461+0000 (oldest deadline 2024-05-08T22:39:32.979401+0000) 2024-05-08T22:39:46.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:45 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8[69891]: 2024-05-08T22:39:45.806+0000 7f094da3a700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.680428+0000 front 2024-05-08T22:39:17.680475+0000 (oldest deadline 2024-05-08T22:39:41.780642+0000) 2024-05-08T22:39:46.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:45 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:45.848+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:46.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:45 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:45.848+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:46.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:45 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:45.848+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:46.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:45 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:45.848+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:46.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:45 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:45.848+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:46.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:46 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:46.358+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:46.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:46 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:46.358+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:46.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:46 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:46.358+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:46.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:46 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:46.358+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:46.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:46 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:46.358+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:46.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:46 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:46.555+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:46.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:46 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:46.555+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:46.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:46 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:46.555+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:46.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:46 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:46.555+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:46.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:46 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:46.555+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:46.823 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:46 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:46.555+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:47.138 INFO:journalctl@ceph.osd.8.smithi079.stdout:May 08 22:39:46 smithi079 bash[89024]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-8 2024-05-08T22:39:47.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:46 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:46.830+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:47.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:46 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:46.830+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:47.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:46 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:46.830+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:47.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:46 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:46.830+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:47.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:46 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:46.830+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:47.632 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:47 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:47.368+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:47.632 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:47 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:47.368+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:47.632 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:47 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:47.368+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:47.632 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:47 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:47.368+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:47.632 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:47 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:47.368+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:47.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.8.service' 2024-05-08T22:39:48.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:39:48.011 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-08T22:39:48.012 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-08T22:39:48.012 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.9 2024-05-08T22:39:48.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:47 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:47.579+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:48.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:47 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:47.579+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:48.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:47 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:47.579+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:48.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:47 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:47.579+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:48.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:47 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:47.579+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:48.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:47 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:47.579+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:48.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:47.807+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:48.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:47.807+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:48.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:47.807+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:48.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:47.807+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:48.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:47.807+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:48.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:47 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:47.807+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:20.957102+0000 front 2024-05-08T22:39:20.957056+0000 (oldest deadline 2024-05-08T22:39:46.856872+0000) 2024-05-08T22:39:48.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:48 smithi062 systemd[1]: Stopping Ceph osd.9 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:39:48.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:48 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:48.345+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:48.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:48 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:48.345+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:48.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:48 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:48.345+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:48.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:48 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:48.345+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:48.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:48 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:48.345+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:48.671 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:48 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:48.255+0000 7f7837165700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:39:48.671 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:48 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:48.255+0000 7f7837165700 -1 osd.9 84 *** Got signal Terminated *** 2024-05-08T22:39:48.671 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:48 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:48.255+0000 7f7837165700 -1 osd.9 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:39:49.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:48 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:48.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:49.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:48 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:48.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:49.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:48 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:48.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:49.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:48 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:48.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:49.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:48 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:48.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:49.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:48 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:48.629+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:49.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:48 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:48.820+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:49.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:48 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:48.820+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:49.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:48 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:48.820+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:49.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:48 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:48.820+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:49.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:48 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:48.820+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:49.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:48 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:48.820+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:20.957102+0000 front 2024-05-08T22:39:20.957056+0000 (oldest deadline 2024-05-08T22:39:46.856872+0000) 2024-05-08T22:39:49.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:49 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:49.297+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:49.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:49 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:49.297+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:49.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:49 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:49.297+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:49.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:49 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:49.297+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:49.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:49 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:49.297+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:50.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:49 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:49.668+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:50.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:49 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:49.668+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:50.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:49 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:49.668+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:50.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:49 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:49.668+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:50.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:49 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:49.668+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:50.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:49 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:49.668+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:50.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:49 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:49.784+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:50.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:49 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:49.784+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:50.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:49 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:49.784+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:50.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:49 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:49.784+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:50.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:49 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:49.784+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:50.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:49 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:49.784+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:20.957102+0000 front 2024-05-08T22:39:20.957056+0000 (oldest deadline 2024-05-08T22:39:46.856872+0000) 2024-05-08T22:39:50.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:50 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:50.336+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:50.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:50 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:50.336+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:50.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:50 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:50.336+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:50.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:50 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:50.336+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:50.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:50 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:50.336+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:51.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:50 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:50.670+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:51.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:50 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:50.670+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:51.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:50 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:50.670+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:51.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:50 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:50.670+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:51.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:50 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:50.670+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:51.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:50 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:50.670+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:51.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:50 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:50.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:51.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:50 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:50.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:51.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:50 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:50.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:51.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:50 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:50.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:51.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:50 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:50.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:51.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:50 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:50.819+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:20.957102+0000 front 2024-05-08T22:39:20.957056+0000 (oldest deadline 2024-05-08T22:39:46.856872+0000) 2024-05-08T22:39:51.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:51 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:51.297+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:51.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:51 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:51.297+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:51.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:51 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:51.297+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:51.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:51 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:51.297+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:51.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:51 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:51.297+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:52.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:51 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:51.624+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:52.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:51 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:51.624+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:52.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:51 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:51.624+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:52.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:51 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:51.624+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:52.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:51 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:51.624+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:52.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:51 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:51.624+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:52.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:51 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:51.791+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:52.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:51 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:51.791+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:52.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:51 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:51.791+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:52.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:51 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:51.791+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:52.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:51 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:51.791+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:52.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:51 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:51.791+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:20.957102+0000 front 2024-05-08T22:39:20.957056+0000 (oldest deadline 2024-05-08T22:39:46.856872+0000) 2024-05-08T22:39:52.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:52 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:52.288+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:52.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:52 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:52.288+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:52.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:52 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:52.288+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:52.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:52 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:52.288+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:52.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:52 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:52.288+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:53.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:52 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:52.610+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:53.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:52 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:52.610+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:53.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:52 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:52.610+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:53.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:52 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:52.610+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:53.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:52 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:52.610+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:53.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:52 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:52.610+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:53.171 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:52 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:52.754+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.748954+0000 front 2024-05-08T22:38:48.748930+0000 (oldest deadline 2024-05-08T22:39:12.849190+0000) 2024-05-08T22:39:53.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:52 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:52.754+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:55.150466+0000 front 2024-05-08T22:38:55.150534+0000 (oldest deadline 2024-05-08T22:39:19.250637+0000) 2024-05-08T22:39:53.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:52 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:52.754+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:00.351835+0000 front 2024-05-08T22:39:00.351918+0000 (oldest deadline 2024-05-08T22:39:23.851843+0000) 2024-05-08T22:39:53.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:52 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:52.754+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:09.553852+0000 front 2024-05-08T22:39:09.553904+0000 (oldest deadline 2024-05-08T22:39:32.954705+0000) 2024-05-08T22:39:53.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:52 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:52.754+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.055942+0000 front 2024-05-08T22:39:17.056088+0000 (oldest deadline 2024-05-08T22:39:39.355750+0000) 2024-05-08T22:39:53.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:52 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9[87207]: 2024-05-08T22:39:52.754+0000 7f782ff54700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:20.957102+0000 front 2024-05-08T22:39:20.957056+0000 (oldest deadline 2024-05-08T22:39:46.856872+0000) 2024-05-08T22:39:53.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:53 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:53.310+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:53.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:53 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:53.310+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:53.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:53 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:53.310+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:53.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:53 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:53.310+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:53.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:53 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:53.310+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:54.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:53 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:53.655+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:54.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:53 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:53.655+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:54.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:53 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:53.655+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:54.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:53 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:53.655+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:54.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:53 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:53.655+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:54.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:53 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:53.655+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:54.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:53 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:53.655+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:31.653641+0000 front 2024-05-08T22:39:31.653561+0000 (oldest deadline 2024-05-08T22:39:53.353486+0000) 2024-05-08T22:39:54.172 INFO:journalctl@ceph.osd.9.smithi062.stdout:May 08 22:39:53 smithi062 bash[113425]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-9 2024-05-08T22:39:54.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:54 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:54.352+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:54.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:54 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:54.352+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:54.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:54 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:54.352+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:54.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:54 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:54.352+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:54.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:54 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:54.352+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:54.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:54 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:54.352+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:39:55.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:54 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:54.680+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:55.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:54 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:54.680+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:55.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:54 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:54.680+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:55.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:54 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:54.680+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:55.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:54 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:54.680+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:55.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:54 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:54.680+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:55.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:54 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:54.680+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:31.653641+0000 front 2024-05-08T22:39:31.653561+0000 (oldest deadline 2024-05-08T22:39:53.353486+0000) 2024-05-08T22:39:55.351 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.9.service' 2024-05-08T22:39:55.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:39:55.393 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-08T22:39:55.393 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-08T22:39:55.393 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.10 2024-05-08T22:39:55.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:55 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:55.313+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:55.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:55 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:55.313+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:55.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:55 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:55.313+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:55.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:55 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:55.313+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:55.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:55 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:55.313+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:55.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:55 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:55.313+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:39:55.714 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:55 smithi066 systemd[1]: Stopping Ceph osd.10 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:39:55.714 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:55 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:55.634+0000 7fe353f89700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:39:55.714 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:55 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:55.634+0000 7fe353f89700 -1 osd.10 84 *** Got signal Terminated *** 2024-05-08T22:39:55.715 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:55 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:55.634+0000 7fe353f89700 -1 osd.10 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:39:56.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:55 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:55.714+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:56.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:55 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:55.714+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:56.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:55 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:55.714+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:56.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:55 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:55.714+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:56.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:55 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:55.714+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:56.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:55 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:55.714+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:56.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:55 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:55.714+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:31.653641+0000 front 2024-05-08T22:39:31.653561+0000 (oldest deadline 2024-05-08T22:39:53.353486+0000) 2024-05-08T22:39:56.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:56 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:56.268+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:56.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:56 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:56.268+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:56.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:56 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:56.268+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:56.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:56 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:56.268+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:56.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:56 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:56.268+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:56.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:56 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:56.268+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:39:57.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:56 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:56.698+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:57.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:56 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:56.698+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:57.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:56 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:56.698+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:57.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:56 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:56.698+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:57.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:56 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:56.698+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:57.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:56 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:56.698+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:57.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:56 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:56.698+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:31.653641+0000 front 2024-05-08T22:39:31.653561+0000 (oldest deadline 2024-05-08T22:39:53.353486+0000) 2024-05-08T22:39:57.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:57 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:57.261+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:57.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:57 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:57.261+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:57.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:57 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:57.261+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:57.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:57 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:57.261+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:57.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:57 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:57.261+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:57.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:57 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:57.261+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:39:58.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:57.687+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:58.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:57.687+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:58.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:57.687+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:58.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:57.687+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:58.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:57.687+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:58.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:57.687+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:58.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:57 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:57.687+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:31.653641+0000 front 2024-05-08T22:39:31.653561+0000 (oldest deadline 2024-05-08T22:39:53.353486+0000) 2024-05-08T22:39:58.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:58.277+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:58.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:58.277+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:58.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:58.277+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:58.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:58.277+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:58.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:58.277+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:58.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:58 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:58.277+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:39:59.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:58.699+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:39:59.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:58.699+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:39:59.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:58.699+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:39:59.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:58.699+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:39:59.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:58.699+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:39:59.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:58.699+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:39:59.074 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:58 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:58.699+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:31.653641+0000 front 2024-05-08T22:39:31.653561+0000 (oldest deadline 2024-05-08T22:39:53.353486+0000) 2024-05-08T22:39:59.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:59 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:59.314+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:39:59.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:59 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:59.314+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:39:59.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:59 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:59.314+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:39:59.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:59 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:59.314+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:39:59.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:59 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:59.314+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:39:59.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:39:59 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:39:59.314+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:40:00.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:59 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:59.747+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.848457+0000 front 2024-05-08T22:38:48.848339+0000 (oldest deadline 2024-05-08T22:39:12.948412+0000) 2024-05-08T22:40:00.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:59 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:59.747+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:52.948784+0000 front 2024-05-08T22:38:52.948892+0000 (oldest deadline 2024-05-08T22:39:18.249103+0000) 2024-05-08T22:40:00.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:59 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:59.747+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.350155+0000 front 2024-05-08T22:39:02.350169+0000 (oldest deadline 2024-05-08T22:39:28.250019+0000) 2024-05-08T22:40:00.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:59 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:59.747+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-05-08T22:39:08.250715+0000 front 2024-05-08T22:39:08.250684+0000 (oldest deadline 2024-05-08T22:39:31.750443+0000) 2024-05-08T22:40:00.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:59 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:59.747+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:17.551938+0000 front 2024-05-08T22:39:17.551965+0000 (oldest deadline 2024-05-08T22:39:43.451979+0000) 2024-05-08T22:40:00.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:59 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:59.747+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.452580+0000 front 2024-05-08T22:39:23.452505+0000 (oldest deadline 2024-05-08T22:39:45.752579+0000) 2024-05-08T22:40:00.073 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:39:59 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10[76719]: 2024-05-08T22:39:59.747+0000 7fe34cd78700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:31.653641+0000 front 2024-05-08T22:39:31.653561+0000 (oldest deadline 2024-05-08T22:39:53.353486+0000) 2024-05-08T22:40:00.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:00 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:00.303+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:40:00.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:00 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:00.303+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:40:00.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:00 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:00.303+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:40:00.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:00 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:00.303+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:40:00.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:00 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:00.303+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:40:00.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:00 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:00.303+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:40:00.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:00 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:00.303+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-08T22:39:34.029122+0000 front 2024-05-08T22:39:34.029160+0000 (oldest deadline 2024-05-08T22:39:59.929109+0000) 2024-05-08T22:40:01.573 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 08 22:40:01 smithi066 bash[87713]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-10 2024-05-08T22:40:01.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:01 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:01.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:40:01.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:01 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:01.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:40:01.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:01 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:01.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:40:01.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:01 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:01.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:40:01.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:01 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:01.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:40:01.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:01 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:01.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:40:01.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:01 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:01.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-08T22:39:34.029122+0000 front 2024-05-08T22:39:34.029160+0000 (oldest deadline 2024-05-08T22:39:59.929109+0000) 2024-05-08T22:40:02.455 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.10.service' 2024-05-08T22:40:02.497 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:02.497 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-08T22:40:02.497 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-08T22:40:02.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.11 2024-05-08T22:40:02.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:02 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:02.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:40:02.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:02 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:02.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:40:02.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:02 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:02.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:40:02.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:02 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:02.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:40:02.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:02 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:02.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:40:02.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:02 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:02.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:40:02.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:02 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:02.312+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-08T22:39:34.029122+0000 front 2024-05-08T22:39:34.029160+0000 (oldest deadline 2024-05-08T22:39:59.929109+0000) 2024-05-08T22:40:02.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:02 smithi079 systemd[1]: Stopping Ceph osd.11 for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:03.138 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:02 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:02.729+0000 7faf3dceb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:03.138 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:02 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:02.729+0000 7faf3dceb700 -1 osd.11 84 *** Got signal Terminated *** 2024-05-08T22:40:03.138 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:02 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:02.729+0000 7faf3dceb700 -1 osd.11 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T22:40:03.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:03 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:03.315+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:40:03.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:03 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:03.315+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:40:03.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:03 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:03.315+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:40:03.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:03 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:03.315+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:40:03.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:03 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:03.315+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:40:03.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:03 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:03.315+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:40:03.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:03 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:03.315+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-08T22:39:34.029122+0000 front 2024-05-08T22:39:34.029160+0000 (oldest deadline 2024-05-08T22:39:59.929109+0000) 2024-05-08T22:40:04.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:04 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:04.269+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:40:04.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:04 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:04.269+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:40:04.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:04 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:04.269+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:40:04.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:04 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:04.269+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:40:04.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:04 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:04.269+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:40:04.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:04 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:04.269+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:40:04.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:04 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:04.269+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-08T22:39:34.029122+0000 front 2024-05-08T22:39:34.029160+0000 (oldest deadline 2024-05-08T22:39:59.929109+0000) 2024-05-08T22:40:05.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:05 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:05.255+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:40:05.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:05 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:05.255+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:40:05.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:05 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:05.255+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:40:05.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:05 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:05.255+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:40:05.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:05 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:05.255+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:40:05.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:05 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:05.255+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:40:05.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:05 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:05.255+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-08T22:39:34.029122+0000 front 2024-05-08T22:39:34.029160+0000 (oldest deadline 2024-05-08T22:39:59.929109+0000) 2024-05-08T22:40:06.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:06 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:06.245+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:40:06.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:06 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:06.245+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:40:06.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:06 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:06.245+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:40:06.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:06 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:06.245+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:40:06.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:06 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:06.245+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:40:06.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:06 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:06.245+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:40:06.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:06 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:06.245+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-08T22:39:34.029122+0000 front 2024-05-08T22:39:34.029160+0000 (oldest deadline 2024-05-08T22:39:59.929109+0000) 2024-05-08T22:40:07.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:07 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:07.237+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-05-08T22:38:48.319603+0000 front 2024-05-08T22:38:48.319567+0000 (oldest deadline 2024-05-08T22:39:09.419748+0000) 2024-05-08T22:40:07.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:07 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:07.237+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-08T22:38:53.520967+0000 front 2024-05-08T22:38:53.520900+0000 (oldest deadline 2024-05-08T22:39:17.620892+0000) 2024-05-08T22:40:07.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:07 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:07.237+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-05-08T22:39:02.122774+0000 front 2024-05-08T22:39:02.122824+0000 (oldest deadline 2024-05-08T22:39:27.422986+0000) 2024-05-08T22:40:07.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:07 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:07.237+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-08T22:39:15.326500+0000 front 2024-05-08T22:39:15.326530+0000 (oldest deadline 2024-05-08T22:39:38.826433+0000) 2024-05-08T22:40:07.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:07 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:07.237+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-05-08T22:39:23.527540+0000 front 2024-05-08T22:39:23.527561+0000 (oldest deadline 2024-05-08T22:39:45.227641+0000) 2024-05-08T22:40:07.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:07 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:07.237+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-05-08T22:39:29.928596+0000 front 2024-05-08T22:39:29.928620+0000 (oldest deadline 2024-05-08T22:39:54.028487+0000) 2024-05-08T22:40:07.639 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:07 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11[78742]: 2024-05-08T22:40:07.237+0000 7faf36ada700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-08T22:39:34.029122+0000 front 2024-05-08T22:39:34.029160+0000 (oldest deadline 2024-05-08T22:39:59.929109+0000) 2024-05-08T22:40:08.638 INFO:journalctl@ceph.osd.11.smithi079.stdout:May 08 22:40:08 smithi079 bash[89289]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-osd-11 2024-05-08T22:40:09.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@osd.11.service' 2024-05-08T22:40:09.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:09.565 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-08T22:40:09.565 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-08T22:40:09.565 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.a 2024-05-08T22:40:09.922 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:40:09 smithi062 systemd[1]: Stopping Ceph mds.a for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:09.922 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:40:09 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-a[95945]: 2024-05-08T22:40:09.811+0000 7f3e679c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:09.922 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:40:09 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-a[95945]: 2024-05-08T22:40:09.811+0000 7f3e679c7700 -1 mds.a *** got signal Terminated *** 2024-05-08T22:40:12.104 INFO:journalctl@ceph.mds.a.smithi062.stdout:May 08 22:40:11 smithi062 bash[113697]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-a 2024-05-08T22:40:12.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.a.service' 2024-05-08T22:40:12.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:12.150 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-08T22:40:12.150 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-08T22:40:12.150 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.d 2024-05-08T22:40:12.419 INFO:journalctl@ceph.mds.d.smithi062.stdout:May 08 22:40:12 smithi062 systemd[1]: Stopping Ceph mds.d for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:12.671 INFO:journalctl@ceph.mds.d.smithi062.stdout:May 08 22:40:12 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-d[95022]: 2024-05-08T22:40:12.419+0000 7fa5c0da8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:12.671 INFO:journalctl@ceph.mds.d.smithi062.stdout:May 08 22:40:12 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-d[95022]: 2024-05-08T22:40:12.419+0000 7fa5c0da8700 -1 mds.d *** got signal Terminated *** 2024-05-08T22:40:14.671 INFO:journalctl@ceph.mds.d.smithi062.stdout:May 08 22:40:14 smithi062 bash[113836]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-d 2024-05-08T22:40:14.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.d.service' 2024-05-08T22:40:14.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:14.757 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-08T22:40:14.757 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-08T22:40:14.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.g 2024-05-08T22:40:15.027 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:40:14 smithi062 systemd[1]: Stopping Ceph mds.g for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:15.421 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:40:15 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-g[95555]: 2024-05-08T22:40:15.026+0000 7fb0c4800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:15.421 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:40:15 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-g[95555]: 2024-05-08T22:40:15.026+0000 7fb0c4800700 -1 mds.g *** got signal Terminated *** 2024-05-08T22:40:19.535 INFO:journalctl@ceph.mds.g.smithi062.stdout:May 08 22:40:19 smithi062 bash[113975]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-g 2024-05-08T22:40:19.541 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.g.service' 2024-05-08T22:40:19.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:19.576 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-08T22:40:19.576 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-08T22:40:19.576 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.j 2024-05-08T22:40:19.834 INFO:journalctl@ceph.mds.j.smithi062.stdout:May 08 22:40:19 smithi062 systemd[1]: Stopping Ceph mds.j for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:20.171 INFO:journalctl@ceph.mds.j.smithi062.stdout:May 08 22:40:19 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-j[94632]: 2024-05-08T22:40:19.834+0000 7f7a05bcf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:20.171 INFO:journalctl@ceph.mds.j.smithi062.stdout:May 08 22:40:19 smithi062 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-j[94632]: 2024-05-08T22:40:19.834+0000 7f7a05bcf700 -1 mds.j *** got signal Terminated *** 2024-05-08T22:40:24.421 INFO:journalctl@ceph.mds.j.smithi062.stdout:May 08 22:40:24 smithi062 bash[114113]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-j 2024-05-08T22:40:24.455 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.j.service' 2024-05-08T22:40:24.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:24.490 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-08T22:40:24.490 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-08T22:40:24.490 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.b 2024-05-08T22:40:24.823 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:40:24 smithi066 systemd[1]: Stopping Ceph mds.b for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:24.823 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:40:24 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-b[83604]: 2024-05-08T22:40:24.730+0000 7fb98b675700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:24.823 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:40:24 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-b[83604]: 2024-05-08T22:40:24.730+0000 7fb98b675700 -1 mds.b *** got signal Terminated *** 2024-05-08T22:40:28.056 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 08 22:40:27 smithi066 bash[87979]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-b 2024-05-08T22:40:28.063 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.b.service' 2024-05-08T22:40:28.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:28.102 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-08T22:40:28.102 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-08T22:40:28.102 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.e 2024-05-08T22:40:28.323 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 08 22:40:28 smithi066 systemd[1]: Stopping Ceph mds.e for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:28.824 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 08 22:40:28 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-e[82921]: 2024-05-08T22:40:28.362+0000 7f1a1de3c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:28.825 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 08 22:40:28 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-e[82921]: 2024-05-08T22:40:28.362+0000 7f1a1de3c700 -1 mds.e *** got signal Terminated *** 2024-05-08T22:40:32.021 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 08 22:40:31 smithi066 bash[88118]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-e 2024-05-08T22:40:32.031 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.e.service' 2024-05-08T22:40:32.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:32.067 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-08T22:40:32.067 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-08T22:40:32.068 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.h 2024-05-08T22:40:32.323 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:40:32 smithi066 systemd[1]: Stopping Ceph mds.h for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:32.823 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:40:32 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-h[83262]: 2024-05-08T22:40:32.329+0000 7fefa21b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:32.823 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:40:32 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-h[83262]: 2024-05-08T22:40:32.329+0000 7fefa21b9700 -1 mds.h *** got signal Terminated *** 2024-05-08T22:40:35.982 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 08 22:40:35 smithi066 bash[88257]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-h 2024-05-08T22:40:35.987 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.h.service' 2024-05-08T22:40:36.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:36.022 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-08T22:40:36.022 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-08T22:40:36.023 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.k 2024-05-08T22:40:36.278 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:40:36 smithi066 systemd[1]: Stopping Ceph mds.k for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:36.573 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:40:36 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-k[82581]: 2024-05-08T22:40:36.278+0000 7f2b0cb99700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:36.573 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:40:36 smithi066 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-k[82581]: 2024-05-08T22:40:36.278+0000 7f2b0cb99700 -1 mds.k *** got signal Terminated *** 2024-05-08T22:40:40.048 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 08 22:40:39 smithi066 bash[88396]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-k 2024-05-08T22:40:40.054 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.k.service' 2024-05-08T22:40:40.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:40.088 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-08T22:40:40.088 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-08T22:40:40.088 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.c 2024-05-08T22:40:40.388 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:40:40 smithi079 systemd[1]: Stopping Ceph mds.c for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:40.388 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:40:40 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-c[84834]: 2024-05-08T22:40:40.330+0000 7fcbce809700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:40.388 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:40:40 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-c[84834]: 2024-05-08T22:40:40.330+0000 7fcbce809700 -1 mds.c *** got signal Terminated *** 2024-05-08T22:40:44.138 INFO:journalctl@ceph.mds.c.smithi079.stdout:May 08 22:40:43 smithi079 bash[89545]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-c 2024-05-08T22:40:44.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.c.service' 2024-05-08T22:40:44.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:44.185 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-08T22:40:44.185 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-08T22:40:44.186 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.f 2024-05-08T22:40:44.445 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:40:44 smithi079 systemd[1]: Stopping Ceph mds.f for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:44.888 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:40:44 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-f[84487]: 2024-05-08T22:40:44.445+0000 7fac48f52700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:44.888 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:40:44 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-f[84487]: 2024-05-08T22:40:44.445+0000 7fac48f52700 -1 mds.f *** got signal Terminated *** 2024-05-08T22:40:48.077 INFO:journalctl@ceph.mds.f.smithi079.stdout:May 08 22:40:47 smithi079 bash[89681]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-f 2024-05-08T22:40:48.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.f.service' 2024-05-08T22:40:48.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:48.119 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-08T22:40:48.119 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-08T22:40:48.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.i 2024-05-08T22:40:48.378 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:40:48 smithi079 systemd[1]: Stopping Ceph mds.i for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:48.638 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:40:48 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-i[85180]: 2024-05-08T22:40:48.378+0000 7feadf59e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:48.638 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:40:48 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-i[85180]: 2024-05-08T22:40:48.378+0000 7feadf59e700 -1 mds.i *** got signal Terminated *** 2024-05-08T22:40:52.076 INFO:journalctl@ceph.mds.i.smithi079.stdout:May 08 22:40:51 smithi079 bash[89829]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-i 2024-05-08T22:40:52.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.i.service' 2024-05-08T22:40:52.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:52.118 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-08T22:40:52.118 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-08T22:40:52.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.l 2024-05-08T22:40:52.376 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 08 22:40:52 smithi079 systemd[1]: Stopping Ceph mds.l for 49d88ea8-0d8a-11ef-bc97-c7b262605968... 2024-05-08T22:40:52.638 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 08 22:40:52 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-l[84138]: 2024-05-08T22:40:52.375+0000 7fdd6d92b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T22:40:52.638 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 08 22:40:52 smithi079 ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-l[84138]: 2024-05-08T22:40:52.375+0000 7fdd6d92b700 -1 mds.l *** got signal Terminated *** 2024-05-08T22:40:56.059 INFO:journalctl@ceph.mds.l.smithi079.stdout:May 08 22:40:55 smithi079 bash[89969]: ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968-mds-l 2024-05-08T22:40:56.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49d88ea8-0d8a-11ef-bc97-c7b262605968@mds.l.service' 2024-05-08T22:40:56.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T22:40:56.098 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-08T22:40:56.098 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T22:40:56.293 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:40:57.754 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T22:40:57.945 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:40:59.421 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T22:40:59.614 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:41:00.996 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:41:01.034 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:41:01.072 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T22:41:01.102 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-08T22:41:01.103 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109/remote/smithi062/crash 2024-05-08T22:41:01.104 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/crash -- . 2024-05-08T22:41:01.149 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T22:41:01.149 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2024-05-08T22:41:01.151 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109/remote/smithi066/crash 2024-05-08T22:41:01.152 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/crash -- . 2024-05-08T22:41:01.188 INFO:teuthology.orchestra.run.smithi066.stderr:tar: /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T22:41:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:tar: Error is not recoverable: exiting now 2024-05-08T22:41:01.191 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109/remote/smithi079/crash 2024-05-08T22:41:01.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/crash -- . 2024-05-08T22:41:01.233 INFO:teuthology.orchestra.run.smithi079.stderr:tar: /var/lib/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T22:41:01.233 INFO:teuthology.orchestra.run.smithi079.stderr:tar: Error is not recoverable: exiting now 2024-05-08T22:41:01.235 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-08T22:41:01.235 DEBUG:teuthology.orchestra.run.smithi062:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-05-08T22:41:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:grep: /var/log/ceph/49d88ea8-0d8a-11ef-bc97-c7b262605968/ceph.log: No such file or directory 2024-05-08T22:41:01.274 INFO:tasks.cephadm:Compressing logs... 2024-05-08T22:41:01.275 DEBUG:teuthology.orchestra.run.smithi062:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:41:01.317 DEBUG:teuthology.orchestra.run.smithi066:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:41:01.320 DEBUG:teuthology.orchestra.run.smithi079:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:41:01.346 INFO:teuthology.orchestra.run.smithi079.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T22:41:01.347 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T22:41:01.349 INFO:teuthology.orchestra.run.smithi066.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T22:41:06.704 INFO:tasks.cephadm:Archiving logs... 2024-05-08T22:41:06.704 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109/remote/smithi062/log 2024-05-08T22:41:06.706 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T22:41:07.246 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109/remote/smithi066/log 2024-05-08T22:41:07.246 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T22:41:07.561 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109/remote/smithi079/log 2024-05-08T22:41:07.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T22:41:08.237 INFO:tasks.cephadm:Removing cluster... 2024-05-08T22:41:08.237 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 --force 2024-05-08T22:41:08.428 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:41:08.680 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 --force 2024-05-08T22:41:08.871 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:41:09.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49d88ea8-0d8a-11ef-bc97-c7b262605968 --force 2024-05-08T22:41:09.300 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: 49d88ea8-0d8a-11ef-bc97-c7b262605968 2024-05-08T22:41:09.535 INFO:tasks.cephadm:Removing cephadm ... 2024-05-08T22:41:09.535 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T22:41:09.557 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T22:41:09.576 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T22:41:09.594 INFO:tasks.cephadm:Teardown complete 2024-05-08T22:41:09.595 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T22:41:09.606 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T22:41:09.606 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T22:41:09.608 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T22:41:09.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T22:41:09.732 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T22:41:09.744 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T22:41:09.744 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:41:09.747 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:41:09.749 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T22:41:09.764 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2024-05-08T22:41:09.765 INFO:teuthology.orchestra.run.smithi079.stderr:bash: ntpq: command not found 2024-05-08T22:41:09.765 INFO:teuthology.orchestra.run.smithi066.stderr:bash: ntpq: command not found 2024-05-08T22:41:09.770 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:41:09.770 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-05-08T22:41:09.770 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +1530us[+1558us] +/- 48ms 2024-05-08T22:41:09.770 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -1384us[-1384us] +/- 55ms 2024-05-08T22:41:09.770 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -558us[ -530us] +/- 70ms 2024-05-08T22:41:09.770 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:41:09.772 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:41:09.772 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-05-08T22:41:09.772 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv01.front.sepia.ceph.com 3 7 377 8 +1475us[+1478us] +/- 48ms 2024-05-08T22:41:09.772 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -1431us[-1428us] +/- 55ms 2024-05-08T22:41:09.772 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -601us[ -598us] +/- 70ms 2024-05-08T22:41:09.772 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:41:09.772 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T22:41:09.773 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-05-08T22:41:09.773 INFO:teuthology.orchestra.run.smithi079.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +1540us[+1527us] +/- 48ms 2024-05-08T22:41:09.773 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 -1383us[-1396us] +/- 55ms 2024-05-08T22:41:09.773 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -554us[ -567us] +/- 70ms 2024-05-08T22:41:09.773 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T22:41:09.773 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T22:41:09.785 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T22:41:09.785 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T22:41:09.797 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T22:41:09.892 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T22:41:09.967 DEBUG:teuthology.orchestra.run.smithi079:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T22:41:10.033 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:41:10.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:41:10.067 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:41:10.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:41:10.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:41:10.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:41:10.130 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T22:41:10.142 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T22:41:10.158 INFO:teuthology.task.internal:Duration was 1612.463451 seconds 2024-05-08T22:41:10.159 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T22:41:10.170 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T22:41:10.170 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T22:41:10.173 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T22:41:10.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T22:41:10.221 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:41:10.222 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:41:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:41:10.686 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T22:41:10.686 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-05-08T22:41:10.686 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T22:41:10.712 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-05-08T22:41:10.712 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T22:41:10.738 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2024-05-08T22:41:10.739 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T22:41:10.761 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T22:41:10.762 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:41:10.764 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:41:10.782 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T22:41:10.854 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T22:41:10.855 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T22:41:10.857 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T22:41:10.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T22:41:10.963 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T22:41:10.975 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T22:41:10.975 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T22:41:11.003 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T22:41:11.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T22:41:11.059 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T22:41:11.071 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:41:11.073 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:41:11.075 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:41:11.098 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-05-08T22:41:11.100 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-05-08T22:41:11.125 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2024-05-08T22:41:11.201 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:41:11.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:41:11.218 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:41:11.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:41:11.236 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T22:41:11.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:41:11.257 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T22:41:11.271 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T22:41:11.272 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109/remote/smithi062 2024-05-08T22:41:11.272 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T22:41:11.317 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109/remote/smithi066 2024-05-08T22:41:11.317 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T22:41:11.358 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698109/remote/smithi079 2024-05-08T22:41:11.358 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T22:41:11.398 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T22:41:11.398 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T22:41:11.400 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T22:41:11.402 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T22:41:11.448 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T22:41:11.463 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T22:41:11.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T22:41:11.478 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T22:41:11.479 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T22:41:11.481 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T22:41:11.483 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T22:41:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 22:41 /home/ubuntu/cephtest 2024-05-08T22:41:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: 262164 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 22:41 /home/ubuntu/cephtest 2024-05-08T22:41:11.504 INFO:teuthology.orchestra.run.smithi079.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 22:41 /home/ubuntu/cephtest 2024-05-08T22:41:11.505 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T22:41:11.522 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T22:41:11.561 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/direct_io}} duration: 1612.4634511470795 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-08T22:41:11.561 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T22:41:11.643 INFO:teuthology.run:pass